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Introduction to International Issues 


Stephen P. Heyneman 
Peabody College of Vanderbilt University 


This issue presents issues in Comparative and International Education and represents the degree to 
which the field has been integrating with issues of domestic concern. Instead of focusing on exotic 
and isolated theories, the articles here demonstrate that issues of concern within OECD countries, 
including the United States, quite often have origins or equivalents elsewhere. For instance, 
the demand for teacher quality, particularly in math and science, is a significant issue in every 
American school district. The article by Kara Brown and Doyle Stevick, “The Globalizing Labor 
Market in Education: Teachers as Cultural Ambassadors or Agents of Institutional Isomorphism,” 
discusses what happens when teachers are imported. They mention that a great deal of attention 
has been paid to the dangers of borrowing policy and insufficient attention to what happens 
when teachers adopt and disseminate practice at the local level. They portray what may become 
common, the existence of a “universal teacher,” one with experience in multiple and divergent 
environments. 

Domestic policymakers often worry about the nature of evidence in education. The trend 
has been to utilize randomized trials as a means to isolate and determine unambiguous effects. 
The use of randomized trials is also an issue in Eastern Africa. The article by Oketch et al., 
“When to Randomize: Lessons from Independent Impact Evaluation of Reading to Learn (RtL) 
Programmme to Improve Literacy and Numeracy in Kenya and Uganda,” draws into question 
the use of randomized trials if the complexities of randomization outweigh its potential virtues, 
a lesson of broad importance. 

Voucher schemes and gaps in achievement are also of high importance, in the United States 
and elsewhere. The article by Jonathan Stern, “The ‘Developing’ Achievement Gap: Columbian 
Voucher Reform,” suggests that the income-contingent vouchers used to overcome achievement 
gaps from an “underfunded, inefficient and/or inadequately supplied public school sector” have 
been genuinely helpful and, with proper improvements, should be expanded. 

How to employ cognitive neuroscience in education has long been a priority within American 
schools of education. The article by Helen Abadzi, “How to Improve Schooling Outcomes in 
Low-Income Countries? The Challenge and Hopes of Cognitive Neuroscience,” informs us that 
using cognitive neuroscience may be even more useful in developing countries. She points out 
that learning outcomes are very low in part because curriculum often overstates student ability 
to “learn complex methods with scant practice.” She includes perceptual learning for scripts, 
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automaticity for basic skills, the limitations of working memory, and the formation of cognitive 
networks in long-term memory as being among the applicable concepts. 

Student cheating and plagiarism continue to be problems in the United States. These problems 
in some ways mirror how corruption may play out elsewhere. Two articles address themselves 
to this set of issues. In her article, “Acting and Reacting: Youth’s Behavior in Corrupt Educa- 
tional Settings,” Amra Sabic-El-Rayess explores the ability of students to cope with exposure to 
educational corruption and the degree to which their private norms can hold sway against what 
appear to be the norms of their peers. Taking her sample from the higher education systems 
in Bosnia and Herzegovina, she employs theories of organizational “voice, exit and loyalty” 
to better understand how students react to the “failing organizations” educating them. In the 
second article on education corruption by Boehm and Caprio, “Fostering Good Governance at 
the School Level in Honduras: The Role of Transparency Bulletin Boards,” the authors describe 
the rather significant effect of a “homegrown response” to increase transparency and improve 
governance. 

Then there is the popular question as to how learning outcomes compare across countries and 
whether previous comparisons have been adequate. The article by Jeongwoo Lee, “An Attempt 
to Reinterpret Student Learning Outcomes: A Cross-National Comparative Study,” argues that 
previous attempts have treated rankings on core subjects such as reading, math, and science as 
though they were the equivalent of educational outcomes. His article compares scores across 
subjects, including civics education, to explore both cognitive achievement and civic knowledge 
achievement. It also has assessed outputs in accordance with the inputs of expenditures and 
learning time. His discussion of these new factors suggests that it might be wise to reevaluate 
the country rankings and reinterpret student performances, taking educational efficiency into 
consideration. 

It is often claimed that “the world is becoming smaller”; communications are faster, interna- 
tional travel is more common and for both economic and political purposes, and immigration is 
now commonplace. However, few have studied what effect such mobility might have. This is the 
subject of the article by Laura Engel, Leslie Rutkowski, and David Rutkowski. They draw on 
the data collected in the study sponsored by the International Association for the Evaluation of 
Education Achievement on Civics Education and find that international mobility may exacerbate 
societal inequalities but that schools may have an important impact on modifying these adverse 
consequences. 

Last, it is sometimes suggested that when those who are educated are also unemployed, 
the likelihood for political protest participation increases. Protests have become a worldwide 
phenomenon (Occupy Wall Street, Arab Spring, Chilean Winter). This study considers the rela- 
tionship between education and protest participation. In the article “Are Student Protests in Arab 
States Caused by Economic and Political Grievances? Empirical Evidence from the 2006-07 
Arab Barometer,” Najeeb Shafiq et al. find that protests in Algeria, Morocco, and Yemen, but 
not in Jordan, are related to their grievances concerning the economy, family poverty, political 
exclusion, and lack of democracy. 

In each of these articles the relevance of Comparative and International Education is reiterated 
to both those interested in local policy reform and those interested in school policy reform 
internationally. 
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The Globalizing Labor Market in Education: Teachers 
as Cultural Ambassadors or Agents of Institutional 
Isomorphism? 


Kara D. Brown and Doyle Stevick 


University of South Carolina 


Institutional isomorphists and other proponents of world culture theory argue that schools around the 
world are converging in many ways, whereas anthropologists and others question this conclusion, 
often arguing that local cultural differences belie superficial similarities. These viewpoints are not 
merely academic explanations of the spread and apparent convergence of education policies and 
practices around the world but are often present in policy and practice. The authors seek both to 
shed new light on these often-entrenched positions and to refocus the debate by considering the 
presence and influence of such views in the policies and practices of international teacher exchanges. 
In the context of the expanding global labor market for teachers, the authors consider the implicit 
theories underpinning international exchange policies and the ways in which the exchange teachers 
themselves make sense of these policies. 

In particular, we recognize that although extensive work has been done on the dynamics of policy 
borrowing, little attention has been paid to international exchange teachers as potential agents of 
isomorphism, adopting and disseminating practices at the local level. Paradoxically, the exchange 
policies construct a universal teacher who is interchangeable across national (and cultural) contexts, 
a view resonant with institutional isomorphists, while justifying the exchanges rhetorically on the 
basis of their value as a cultural exchange, a view more consistent with the culturalists. The teachers 
who participate, however—and who effectively self-select by their beliefs that such exchanges are 
possible—accept the interchangeability thesis and view such exchanges as a professional development 
opportunity. 


The globalization of labor markets, well-established in nursing (Ross, Polsky, & Sochalski, 2005), 
many technology-intensive professions, and higher education (Musselin, 2004), has expanded 
rapidly in American schools. Although overseas-trained teachers (OTTs) have worked in “foreign” 
contexts for decades (see, e.g., Abir, 1970, on the recruitment of Indian teachers to Ethiopia), the 
United States now joins countries ranging from the United Kingdom (Commonwealth Secretariat, 
2004) to Botswana (Appleton, Sives, & Morgan, 2006) in looking abroad for teachers to address 
critical shortages. Over the last 15 years, a wide array of public and private agencies—including 
the U.S. Department of State, state departments of education, local school districts, and nonprofit 
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and for-profit teacher recruitment companies—have sought international teacher exchanges in 
order to address persistent and growing teacher shortages. Although K-12 schools across the 
United States employ OTTs, concentrations can be found in California, major urban centers (e.g., 
New York City, Washington, DC, Baltimore) and inthe Southeast. 

The emerging globalization of the labor market for teachers has profound theoretical and prac- 
tical significance. Theoretically, the practice of exchanging teachers across national contexts can 
shed new light on calcifying debates between world culture theorists (or institutional isomorphists, 
as they are sometimes called) and their critics. Practically speaking, educators and policymakers 
must deal with ethical concerns (e.g., the risk of brain drain) and cultural challenges (what kinds 
of special preparation are required for teachers to function effectively cross-culturally?). 

Research on OTTs is in its infancy. The valuable studies that do exist (examples include 
Barber, 2003; Dunn, 2011, 2013; Miller, Mulvaney, & Ochs, 2007) come from a variety of 
methodological and conceptual approaches. Basic data regarding international teacher exchange 
policies and practices, including the distribution of teachers, their specializations, and countries of 
origin, are lacking. In addition, we know little about the merits and problems of these exchanges 
and less about the experiences of teachers in them. In this article, we seek to address these deficits 
by reporting on the first two phases of a long-term research project on the policy and practice of 
international teacher exchanges in the U.S. state of South Carolina. 

We first consider the practice of international teacher exchanges in relation to the broader 
debates between those who believe that schools around the world are converging in form, orga- 
nization, and practice (the world culture theory or institutional isomorphist position) and critics 
who argue that the appearance of convergence stem from superficial similarities that disguise ed- 
ucation’s fundamentally cultural, situated and contextual character. We then outline the broader 
project’s research design and methods. The first phase of the project shared here includes a de- 
scriptive study of the demographics of teacher exchanges in South Carolina and an analysis of 
the forces that led to its widespread adoption in the state. The second phase shifts the focus to 
understanding the experiences of OTTs who participated in the program. A discussion concludes 
the article. 


THEORETICAL FRAMEWORK 


“Institutional isomorphism,” the theory of global convergence in education, implies that univer- 
sally applicable best practices can be identified and that teachers are interchangeable across na- 
tional contexts (Meyer, Boli, Thomas, & Ramirez, 1997). The contrasting cultural model, rooted 
in anthropological perspectives, asserts that similar structures and practices nevertheless have 
different meanings in different contexts, implying that education policy and teacher preparation 
require a great deal of local knowledge and cultural awareness to be effective (Anderson-Levitt, 
2003). 

For decades, researchers in comparative education have explored the cultural dimensions of 
teaching (Alexander, 2001; Anderson-Levitt, 2002; Shimahara & Sakai, 1995; Stigler & Hiebert, 
1999). Broad agreement exists in the field that teaching is a deeply cultural practice and is 
tightly intertwined with broader societal philosophies and beliefs. Stigler and Hiebert (1999) 
argued that “systems of teaching, because they are cultural, must be understood in relation to the 
cultural beliefs and assumptions that surround them” (p. 88). The cultural dimension of teaching 
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and school practice, however, is often an implicit, taken-for-granted, and invisible aspect of the 
profession that many teachers and others may not recognize in their original contexts. 

As Florio-Ruane (2002) noted, however, teachers are key actors in a “complex ecology” that 
involves “the nature of knowledge, the child’s growth and development, many ways of behaving 
and making sense among diverse citizenry, and the unequal distribution of wealth and power” 
(pp. xi-xii). In an era dominated by the nation-state, in which the borders of culture, language, 
country, and education policy closely align, researchers have examined the cultural knowledge 
needed to teach in countries like France (Anderson-Levitt, 2002), Russia (Alexander, 2001), 
and Japan (Shimahara & Sakai, 1995). According to anthropologists, the deep cultural roots of 
teaching and schooling lead to local variation of teaching and schooling and widely differing 
teacher practices (Anderson-Levitt, 2003). 

Given the cultural aspects of teaching, global teacher migration suggests a possible cultural 
displacement as teachers move away from their home countries, schooling, training, certification, 
and professional experiences and work in foreign contexts. Although this adjustment may be 
complicated, some comparative education scholars argue that a world culture has emerged in 
which the structures, curricula, and goals of schools have converged (Baker & LeTendre, 2005; 
LeTendre, Baker, Akiba, Goesling, & Wiseman, 2001; Meyer et al., 1997); this convergence di- 
minishes country-to-country differences in practice, and would therefore facilitate OTT practice 
in a new national context. Recent, cross-national research in teacher education on teacher prepa- 
ration for secondary mathematics, for example, found a degree of homogeneity in curricular 
content (Blomeke, 2012). Even critics of institutional isomorphism acknowledge that “educa- 
tional systems, processes and values to appear to be coinciding” (Carney, Rappleye, & Silova, 
2012, p. 387). Beyond similarities in content, world culture theorists underscore the “global cul- 
tural dynamics to draw attention to the fact that national systems of education around the world 
are affected by a dominant global culture that has a specific content (i.e., values of rationalization, 
bureaucracy, individualization, democratic governance)” (LeTendre, Baker, Akiba, Goesling, & 
Wiseman, 2002, p. 22). 


METHODOLOGY 


The research drawn on for this article is part of a multiyear, multisite project that began in 
2009 and will run through 2013. In the first phase of the project (2009-2012), we examined 
institutionalized moments of the OTT exchange process: recruitment, selection, induction, and 
the creation of “successful” OTTs. We conducted 22 semistructured and open-ended interviews 
with state department of education officials, representatives from OTT broker companies, and 
South Carolina school administrators in 2009, 2010, and 2011. A key purpose of the first phase 
of data collection was to understand the scope of the use of J-1 teachers at the state level. Due 
in part to the decentralized nature of education in the United States, a national database of 
international teachers is not available. Although we have a sense of the number of OTTs working 
in the United States as a whole, around 20,000 in 2009 (American Federation of Teachers, 2009), 
select state-based compilations are generally unavailable (for an exception, see Bartlett, 2009, on 
California). To grasp the scope of J-1 teacher placement and global migration flows, we analyzed 
more than 1,000 international teacher files from the SC teacher certification database. We charted 
and mapped the data using Geographic Information Systems with the assistance of geography 
colleagues. In our analysis, we incorporated information from 2006, the year the South Carolina 
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Department of Education (SCDE) began to identify the host J-1 program for each teacher, through 
2009. We are currently in the process of collecting and mapping the next set of data (2010-2013). 
In this article, we share our findings from the 2006-2009 years with the recognition that there 
has been a decline in the number of OTT working im the state. The systematic review and use of 
Geographic Information Systems pointed to the scope of the use of these teachers and included 
their profile (e.g., years of experience, country of origin, degrees), geographic concentration in 
the state, academic fields, and the general trends in migration and placement. 

The interview data we analyze in this article comes from the second phase of the project 
(2012-2013), in which we attempt to understand this policy process from the OTTs’ per- 
spective. Involving teachers at all phases of the process—recent recruits, active hires, and 
returnees/homecomers—we began the teacher interviews with the returnees in order to gain 
a holistic understanding of their J-1 process. Given the long and particular relationship that South 
Carolina has had with Romanian teachers (in part due to an SC teacher-broker company that 
works primarily with Romanian teachers, which results in Romania being the third largest OTT 
sending country to South Carolina), as well as one of the researcher’s expertise in Romania, we 
chose Romanian returnee teachers for the first round of interviews. Focus on teachers from one 
country reveals a longitudinal view on educational development and change within the sending 
country. In the spring of 2013, the researchers will conduct similar interviews in India (the primary 
OTT sending country to South Carolina). 

Our analysis of teacher certification files revealed that the majority of Romanian OTTs came 
from one city—Cluj Napoca. Based on this information and the connections established through 
Phase | interviews, Stevick traveled to Romania and conducted 14 semistructured interviews 
lasting approximately 2 hr each with returnee Romanian OTTs who had worked in South Carolina. 
For the purposes of this article, we selected two cases to illuminate broader trends found across 
the group. We gave these two OTT returnees pseudonyms: Victor and Christina. 


THE BROAD ADOPTION OF INTERNATIONAL TEACHER EXCHANGES 
IN SOUTH CAROLINA 


South Carolina’s broad involvement with international teacher exchanges began only once federal 
policies enabled them. The most important federal policy for enabling such exchanges has been 
the J-1 visa program. South Carolina has hosted more than 2,000 OTTs on 3-year J-1 exchange 
visas since 1998. In purpose, the J-1 program is cultural; professionals work on a temporary (up 
to 3 years, now extendable for | year) basis for the purpose of cultural exchange. According to 
the U.S. Department of State (n.d.), the goal of the Exchange Visitor Program is to “promote 
mutual understanding between the people of the United States and the people of other countries 
through educational and cultural exchanges.” Likewise, the SCDE presents a cultural rationale 
for developing Memoranda of Understanding (MOUs) with other countries to foster teacher 
exchange: “These MOUs have been established as a means of broadening cultural understanding, 
fostering the exchange of educational ideas and techniques, and introducing teachers from other 
countries into the classrooms and communities in South Carolina” (SCDE, n.d.). In their rhetoric, 
the emphasis upon culture aligns well with the anthropological critics of world culture theory. 
The philosophical underpinnings for the J-1 visa for teacher applicants seem directly con- 
tradictory to those undergirding its eligibility requirements. Indeed, the rationale aligns closely 
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with the anthropological critics of institutional isomorphism, whereas the criteria for the visa 
could have been derived from world culture theory. Together, they reveal the paradoxical ways 
in which the U.S. government, districts, and schools invoke the cultural differences that inter- 
national teachers bring while operating on a premise of universal preparation and expertise that 
make teachers effectively interchangeable across national borders. The eligibility requirements 
for the J-1, also called the Exchange Visitor Program, include 3 years of teaching experience (in 
their home country or elsewhere), certification (in their home country), and English proficiency 
(though this is less important for language immersion teachers). The regulations frame experi- 
ence, training, and certification as functionally equivalent, despite cross-national differences in 
these three professional domains. 

The U.S. Department of State oversees the process by approving sponsors to manage the 
recruitment and placement of OTTs. In South Carolina, the SCDE and four private, for-profit 
“teacher broker” firms—Visiting International Faculty, Foreign Academic & Cultural Exchange 
Services (FACES), the Amity Institute, and International Teacher Exchange Services—handle 
the majority of the international placements. The SCDE, which organizes teacher exchange 
programs based on bilateral, Memorandum of Understanding agreements, has established links 
with five countries—India, France, Spain, China, and Taiwan. The financial dynamics of the 
J-1 arrangement differ significantly based on the sponsor. With public agencies like the SCDE, 
teachers and their SC host school/district pay neither recruitment nor placement fees. Private 
companies, however, generally charge teachers or their host schools/districts either a one-time 
placement or an annual fee. Sponsor organizations typically coordinate orientation and in-service 
training for the J-1 teachers in their program in addition to any programs offered by the host 
school or district. 

Although a number of circumstances made SC particularly receptive to the policy—including 
its lack of an effective teacher union—four primary developments in education fed SC’s demand 
for the global recruitment of teachers. First, by passing the No Child Left Behind Act of 2001, the 
federal government increased pressure on school administrators to hire highly qualified teachers. 
Former SC State Superintendent of Education Jim Rex drew attention to the relationship between 
the unmet needs of SC public schools and the recruitment of OTTs: 


We are getting dangerously close in some parts of our state to having a demoralized and compromised 
teaching force. ... In many areas of our state, over the last seven or eight years, we have been 
increasingly hiring foreign teachers, teachers from other countries, India, Romania, and elsewhere 
because we cannot find U.S. citizens and South Carolinians certified to teach in those areas. (University 
of South Carolina, Witten Lecture, 1 October 2007) 


Indeed, many schools found that it was easier to certify international teachers as highly qualified 
than it was to find domestic teachers who met the criteria. 

Second, a persistent teacher shortage existed in many pockets of the country, particularly 
afflicting the most high-needs schools in rural and urban areas. The use of international teachers, 
what the National Education Association once claimed to be “a temporary solution to a long-term 
problem,” has transformed into a sustained approach to address widespread teacher shortages. 
Accordingly, the SCDE advocates the recruitment of international teachers as one method of 
bringing highly qualified teachers to meet.“critical needs in hard-to-staff geographical and subject 
areas” (SCDE, 2006, 17). The same slate of hard-to-staff subjects—special education, math, 
science, foreign language, and English as a Second Language—is found in states like South 
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Carolina, Georgia (Dunn, 2011, 2013), California (Bartlett, 2009), and Arizona (Terrazas & Fix, 
2009). 

The third factor leading to the increased use of international teachers emerges from the 
heightened national concern to develop a globally: competent citizenry and workforce. Both 
the federal and state departments of education assert that temporary teacher exchanges help 
to internationalize public education by bringing the “world to schools.” The U.S. Department 
of Education explicitly voiced its commitment to the use of international teacher exchange 
programs as a way to complement this 21st-century priority of the federal government (Paige, 
2003). Superintendent Rex, upon signing a teacher exchange agreement with France, echoed 
the U.S. Department of Education’s position on the use of international teachers: “I believe that 
the cultural focus for the exchange of ideas will go a long way toward ensuring that our young 
people are able to compete successfully both here and around the globe” (SCDE, 2007). Similarly, 
the SCDE’s (n.d.) description of the International Visiting Teachers Program frames the use of 
OTTs as an opportunity for global enrichment: “School districts throughout the state have an 
opportunity to provide students with programs that are linguistically and culturally rich to better 
prepare them for future success in their personal, academic, and professional lives.” 

Fourth, education faces difficult budgetary pressures, and the financing of international teachers 
is often cheaper than domestic teachers. South Carolina revenues plummeted with the economic 
crisis, yet Governor Sanford attempted to reject federal stimulus money to keep teachers employed 
(he was eventually forced to accept it). Despite this order, hundreds of teachers lost their jobs. 
International teachers offer an efficient means of hiring. As districts have to pay significant 
sums beyond salaries toward domestic teachers’ insurance and retirement, which don’t have to be 
provided for international teachers, they can provide the same salary, pay (in some cases) $10,000 
to an agency, and still cut costs. 

Together, these developments have led to an increasing number of OTTs currently teaching in 
South Carolina’s public schools. In the 2011/2012 schools year, 265 OTTs worked in the state 
(down dramatically from 768 in 2008/2009). Although the trend to hire international teachers, 
especially to fill critical vacancies, began in South Carolina in the late 1990s, the percentage 
of these teachers hired grew exponentially from 2000 until 2008, with a decline starting in the 
2009/2010 academic year due to the recession and budget cuts. The SC Teacher Salary Schedule 
Revision Task Force (2006) observed that the number of OTTs hired in South Carolina increased 
by a factor of 10 from 2003 to 2005 (p. 55). 

J-1 OTTs come to South Carolina from 39 different countries with India, the Philippines, 
Romania, Colombia, and Jamaica representing the primary “sender” countries. The international 
teachers working in South Carolina in 2008/2009 tended to follow a South—North migration 
pattern; that is, they hail primarily from countries located south of the equator, like India. 
Romania, an east European sending country, proves to be an exception to this trend. In addition 
to being developing countries in the South, three of the five the biggest “sender” countries for 
South Carolina—India, Philippines, and Jamaica—also recognize English as an official language. 
Teachers recruited from these countries are accustomed to using English as their professional 
language. The OTTs’ professional English ability is a key factor in facilitating their global 
migration to a new teaching context. 

In South Carolina, as in other states, teacher recruitment and retention are toughest in “hard-to- 
staff” poor, predominantly minority school districts. In SC, the coastal south, the southern corner 
of the state, and the I-95 corridor remain stubbornly difficult regions to staff. Teacher turnover rates 


10 K. D. BROWN AND D. STEVICK 


for 2011 in various school districts reflect these challenges: 37.7% in Bamberg 2, 36.4% in Lee, 
and 22% in Florence 4. J-1 educators teach in schools across the state of South Carolina (in 51 of 
the state’s 81 school districts), but in many cases, the lowest performing school districts turned to 
foreign teachers to fill their teaching shortages and recruit “highly qualified” teachers. Richland 
1 and Williamsburg | hosted the greatest percentage of J-1 teachers—17%—in 2008/2009. 
Although the relative proportions of J-1 teachers to district and school sizes are currently being 
determined, some evidence points to a concentration of J-1 teachers as new and renewed hires in 
particular districts and schools. For example, in Ridgeland Middle School in Jasper County, the 
percentage of foreign teachers constituted as much as one third of the faculty (“Program,” 2004). 
Finally, a geographic pattern has emerged between certain host districts and sender countries. Six 
districts only employ teachers from India (i.e., Williamsburg 1, Marion 2, Beaufort, Marlboro 1, 
Lee 1, and Allendale 1). Other districts tend to draw largely from the same regions. For example, 
Richland | hosted J-1 teachers from India and Colombia (65 out of 85). 

OTT in South Carolina overwhelmingly teach hard-to-staff subjects—particularly secondary 
mathematics, science, foreign language, and special education. In 2008-2009, the greatest per- 
centage of J-1 teachers (31%) taught special education, with math (24%) representing the second 
most-commonly taught subject. The Williamsburg School District alone had 46 teachers from 
India working in special education, math, and science. Data from the state certification files 
suggest that districts differently access the J-1 teacher market with the wealth of the district, as 
measured in part by free and reduced-price lunch rates, playing a role. For example, in Greenville 
1, the ninth wealthiest district in the state, more than 80% of the district’s J-1 educators taught 
a foreign language. In Williamsburg 1, the second poorest district in the state, 58% of the J-1s 
taught special education and 23% taught math. These district-based J-1 profiles point to a differing 
demand across the state with wealthier districts seeking primarily foreign-language J-1 teachers 
and high-poverty, lower achieving districts hosting J-1s certified in math, science, and special 
education. 

Finally, data reveal that in 2008-2009, J-1 teachers had an average of 9.5 years of experience, 
80% were recognized as “highly qualified,” more than 40% of the J-1 teachers had a master’s 
degree or beyond, and 11% of the J-1 teachers returned to South Carolina for a “second tour” in 
the program. In practice, a second tour means that a J-1 teacher completed the first 3 years on 
the visa, returned home for the required period, then re-enrolled in the J-1 program. The greatest 
number of these returnee teachers come from India (n = 32), Romania (n = 19), and Colombia 
(n = 10) and typically return to their original host schools. Thirty-four percent of these J-1 
educators teach special education and 22% teach a foreign language. The returnee rate among 
current J-1 teachers points not only to the sustained need of certain schools within specific 
subject areas and to the possibility of long-term international relationships developing within the 
restrictions of a short-term visa program but also the consistent interest among certain OTTs to 
take their skills global. 


AGENTS OF CONVERGENCE? 


Several key findings emerged from our analysis of the Romanian returnee OTT interviews that 
contribute to the dialogue between the world culture theorists and those who advocate for a more 
prominent role for culture. In this section we share data that attest to the reasons and the ways 
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returnee teachers function as key actors who carry and disseminate pedagogical practices across 
borders. Broadly speaking, the interviews illustrated the ways these OTTs define themselves as 
highly competent professionals, who could work seamlessly in a new national context, but still 
learned new approaches and techniques within their subject area to be applied “at home.” This 
strong sense of professional identity allowed the OTTs to frame their experiences teaching in 
South Carolina as professional development. 

In this following section, we explore Victor and Christina’s reflections on lessons learned about 
(a) student-teacher relationships, (b) discipline and classroom management, and (c) pedagogy 
during their OTT experiences. We then consider the teachers’ identification of places where 
exchange of these “lessons learned” unfolds: in faculty meetings and the classroom. In discussing 
both the lessons learned and the moments of dissemination, we were attentive to the cultural 
explanations the teachers provided for their adoption and advocacy of certain strategies observed 
in the United States. Teachers, as is clear next, are active cultural agents observing, experimenting, 
and selectively taking up new things. 

Victor is a high school math teacher with 39 years of experience. He has taught in multiple 
sites outside his home country—in France (1993) on a government-exchange program and in 
South Carolina (1996 and 1997) through the FACES program.! Victor returned to teach in an 
elite Romanian high school in Cluj. Christina is an elementary school teacher with 11 years of 
experience. She taught special education in a South Carolina middle school through the FACES 
program (2001-2003). Christina returned to Romania in 2003 to pursue her doctorate in special 
education. 


Student—Teacher Relationships 


Returnee Romanian OTTs were struck by the different nature of student-teacher relationships 
in the United States as compared with those in their home country. Teachers like Victor and 
Christina observed the ways teachers in the United States had more “horizontal” than “vertical” 
relationships with students and parents. Teachers in the United States were no longer central 
authority figures but were expected to develop reciprocal relationships with students. 


It’s a European style. The teacher is the magister and the teacher is still the most important person in 
the learning process ... in France, which is similar to Romania, but a bit more open. In the United 
States, it is different. The teacher was more close to the students, employed and paid by the budget 
to teach the students. The relationship was more, more tight and you felt like you were directly 
responsible for this and this and this student. And, when you had the meet with the parent, the 
principal and the teacher, we discussed like partners not like magister and student.—Victor, March 
2012 


'In our research, we have encountered teachers who rotate between countries, as many teachers in international 
schools are known to do. The phenomenon we have come to call “teacher nomads” describes teachers who teach 
primarily domestic children in the (usually) public schools or (sometimes) private schools of countries around the world. 
They may return to their home country to work infrequently. Victor did not meet this threshold, as he maintained his 
permanent residence in his home country and went abroad for two stints of just a few years before returning home for 
extended periods. Teachers in international schools do not generally meet this threshold either, as they are not adapting 
as extensively to domestic educational practices and cultures. 
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Victor’s attention to broad continental (i.e., “European”) style of the teacher’s role and to regional 
similarities and differences reveals the way he triangulates (across French, Romanian, and U.S. 
contexts) rather than dichotomizes difference. Also, he was attentive to the nature of the discourse 
and interaction among himself, the parent, and the principal, who engaged “like partners.” Victor's 
observation of the prevalence of the “partnership” in U.S. schools echoes in anthropological 
research that points to this cultural construction as a dominant model and paradigm for parental 
involvement in schools (Doucet, 2011; Lopez, 2001). 

Christina also began to appreciate new, transformed roles for herself as a teacher resulting 
from her South Carolina experiences. 


In Romania, we have this competition between the students and I didn’t see it very well when I left 
Romania. But, in the U.S., I began to understand that this competition is with the student first. The 
student has to be content with himself or herself; it has to be better than yesterday. So, when I came 
back [from the U.S.], I said to the students, ““You can do more than that. It doesn’t matter what the 
others do. You have to do better than yesterday” and they understood this.—Christina, March 2012 


In highlighting the value of the comparative teaching experience—‘I didn’t see it very well when 
I left Romania”—she explains the way she observes everyday Romanian student competition in 
a new way. As a returnee teacher in Romania, Christina now positions herself as a teacher who 
encourages the students to do their individual best. Of importance, students are receptive to this 
advice—‘‘they understood this”—and her new teacher practice. 


Discipline and Classroom Management 


Romanian returnee OTTs repeatedly noted that their U.S. experiences exposed them to new 
notions of discipline and classroom management. Teachers framed classroom management as 
a culturally neutral practice that may be present in one context but not perceived as culture or 
context specific—it can be picked up and brought back (“This is practical and it works”). 


From my U.S. experience, I learned a very important thing—how to manage the classroom ... 
classroom management. This is very well developed in the U.S. and I have learned a lot about this 
and have applied it in Romania. And, now, at the present, I am working with things that were not 
necessary several years ago in Romania, but are now. We need these things; they are necessary. 
Classroom management, rules, discipline, consequences. And, I have my own rules in my classroom, 
the students know the consequences, sign a contract. And, I’m working like I did in the U.S. ten years 
ago. This is practical and it works.— Victor, March 2012 


Victor’s temporal focus on the arc of educational development in Romania vis-a-vis the United 
States (“I am working with things that were not necessary several years ago in Romania” and “I’m 
working like I did in the U.S. ten years ago”) points to the emergent moments of convergence in 
practice; only as a result of shifts in educational norms and values in Romania has Victor had to 
tap into his reservoir of strategies learned while working in the United States. 
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Pedagogy 


Romanian returnee OTTs developed new insights into pedagogy as a result of their experiences. 
Teachers pointed to the differences in pedagogical approach and curriculum in the United States. 
In some instances, teachers voiced frustration that one nation’s norms and school standards 
created barriers to borrowing and adapting. Victor mentioned that although he appreciated giving 
students the opportunity to use calculators in U.S. math classes, this practice was forbidden in 
Romania. In other cases, examples of pedagogical convergence were highlighted. 


In France and in the United States, [in math class] there are a lot of applications in real life compared to 
Romania. Here [in Romania] math is more abstract, there is more theory and fewer practical problems 
to show students why they need to learn mathematics and where they can apply it.... The necessity of 
learning something—why should I learn, let’s say, functions. There were examples taken from reality, 
then, they define the notion, and go back and apply. Now, in Romania, we also started to connect 
mathematics more to lived reality than before, but still we need to do more about this —Victor, March 
2012 


Victor’s comment points to the logic he finds in this shift (“we need to do more of this”) and 
the perception of convergence. He also identifies a national “we” charged with continuing this 
reform, though teachers endorse general shifts in subject-specific pedagogy. 

Other teachers pointed to the difference in teacher-preparation curriculum and the active role 
they assumed in transforming their practice based on U.S.-based learning. 


Here, after we finish the university, we don’t have some courses to help us with individual education 
plans, accommodations, modifications of the curriculum. This is a big issue here in Romania, so it 
helped me a lot to have this. When I came back to Romania, I followed what I learned in the U.S. I 
applied the information in my classes.—Christina, March 2012 


In this quote, Christina frames the lack of special education teacher training components typical 
in the United States as a “big issue” in Romania. As with Victor in his previous quote about 
bringing classroom-management strategies from the United States to Romania, Christina also 
actively transfers, assumingly without problem, the approaches that are helpful to her to “home” 
context. 

Finally, the Romanian returnee OTTs noted key places and moments when they exchanged 
ideas with their colleagues and students. In their reflections, they noted the spaces and places 
of pedagogical transfer. Several teachers mentioned sharing “lessons learned from teaching in 
the U.S.” in faculty meetings. They recalled that upon their return from the United States (and 
France), colleagues expressed tremendous interest in their experiences and the nature of U.S. 
schooling. With students, teachers strategically deployed cultural sharing at particular times for 
pedagogical ends (e.g., for reward): 


[The students] asked me a lot of times about my life before the fall of communism and we discussed a 
lot about this. ... We had some moments to discuss not during class time and I gave this as a reward. 
{ told them, “If you are doing your work, on Friday in the last ten minutes we can discuss [this], you 
can ask me questions.” And, they did. They asked me about school in Romania, about my life. This 
happened in France and America.—Victor, March 2012 


Victor’s strategy of “culture exchange upon performance” echoes those shared with us by other 
OTTs from Kenya and Russia. These pedagogical moments reveal not only the common (and 
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persistent) student interest in learning about global culture but also the ways OTTs use cultural 
difference as an incentive, a tool, and a reward in the school setting. 


DISCUSSION AND CONCLUSION 


The research reported in this article focuses on the rapid globalization of the teaching force of 
South Carolina and, more broadly, the United States. Since 1998, approximately 2,000 inter- 
national teachers have taught in the state’s public schools. The majority of these J-1 teachers, 
who come to South Carolina largely from developing countries in the global South, are placed 
in high-need schools and teach hard-to-staff subjects like math, science, special education, and 
foreign languages. According to certification data, the globalization of the teacher market has 
led, in part, to a flow of credentialed and experienced teachers to many of South Carolina’s most 
needy public schools. 

Despite their overwhelming placement in “hard-to-staff” schools, the Romanian returnee OTT's 
profiled in this article presented their experiences in U.S. schools as opportunities for professional 
development rather than experiences that were challenging due to cultural differences. The 
teachers’ confidence and pride in their professionalism allowed for a permeability of state and 
cultural boundaries. Victor and Christina communicated their understanding that many teaching 
strategies were applicable and appropriate to any classroom, not just those in the United States. 
Moreover, their time teaching in the United States was one that improved their professional 
repertoires. Thus, the returnee teachers perceived a certain universality to curriculum, pedagogy, 
and teacher practice, in general. These teacher understandings reflect an understanding of global 
homogeneity and serve to converge teacher practice. 

The data also suggest a distinct direction to the exchange and transfer around pedagogical 
strategies—the global transfer was largely unidirectional. As the interview data illustrate, Ro- 
manian returnee OTTs borrowed techniques and strategies from the United States but were not 
positioned or pursued in U.S. schools as teachers who could help transform teacher practice in 
South Carolina. None of the teachers reported schools capitalizing on their presence to learn 
about pedagogy and curriculum in Romania. In a time when a great deal of educational literature 
explores what can be learned from international contexts to improve teacher training, quality, and 
practice (Akiba & LeTendre, 2009; Darling-Hammond & Lieberman, 2012), the “unidirectional 
transfer” is an everyday, missed opportunity for U.S. schools to learn from foreign professionals. 

Finally, returnee Romanian OTTs engage in social practices that produce new meanings of 
space, territory, and boundaries around teaching. Qualitative research helps us to understand 
their rationales and understandings of professionalism, universal subject matter, and appropriate 
pedagogy (or teacher training). These teachers help to create and support notions of the universal 
math or special education teacher and work as teachers who contribute to global convergence 
through the refining of practice in their home country. 

These conclusions are limited, of course, to one country-specific group of returnee OTTs. The 
perspectives are valuable in providing us with a window into the ways that one cultural group 
positions itself in a global system of teacher exchange. How might the insights of Romanian 
teachers at different stages in the exchange process (i.e., recent recruits or in-service teachers) 
differ from those shared by returnees? Will the perspectives of OTTs from India align with those 
from Romania? How do we explain the ways these teachers make sense of their J-1 experiences? 
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Are those explanations themselves deeply rooted in national culture and history or will we find 
the emergence of a universal teacher? We expect that these questions will begin to be addressed 
as we and others continue research on this topic. 
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In East Africa, there is great effort directed toward ensuring that there is learning and value for 
money invested in universal education policies initiated over the past decade. Kenya and Uganda 
are two countries that typify this effort. The effort includes the work of research organisations such 
as Uwezo, which assess learning levels; RTI, which assesses language and early grade reading; and 
the work of African Population and Health Research Center (APHRC), which looks at schooling 
patterns of different households. All these studies paint a disappointing picture both for the universal 
access Education for All policy and the large sums of money that have been devoted to achieve it. 
The verdict is that too many pupils in schools are not learning and too many poor ones are excluded 
from universal public access. Uwezo reports that 70% of pupils in Grade 3 cannot do Grade 2 work, 
and 9% of those completing Primary 8 in Kenya cannot do Grade 2 work. Answers are not easy to 
find, so borrowing what has worked elsewhere has been attempted. This article presents one effort 
toward finding what can work to improve learning for pupils in early grades. It is based on the idea of 
“Reading to Learn” implemented elsewhere and attempted in East Africa by Aga Khan Foundation 
and independently evaluated by APHRC using randomization methods. Lessons presented highlight 
the role and complexities of randomization in addressing the educational challenges in East Africa. 


INTRODUCTION 


As noted by Banerjee and Duflo (2011) in their interesting book Poor Economics, answering the 
big philosophical questions about poverty and what works and what does not work by simple 
country comparisons has not yielded evidence that inspires confidence. The consequence is that 
there is no agreed-upon solution to most challenges of development. They strongly market and 
advocate randomized control trial as the “gold” standard solution to the long-standing debatable 
issues. One such debatable issue is how to get the majority of learners already in classrooms, 
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particularly in poor, developing countries that have recently embraced the idea of universal 
primary education, to be able to read and do math competently at the appropriate grade level. 
There are no simple solutions, so numerous decisions are made along the following lines: increase 
textbooks, increase opportunity to learn, reduce class sizes by hiring more teachers, hold teachers 
accountable, improve school management, train teachers, and so forth. None of these priorities 
seem to have led to a straightforward answer or solution, and few produce evidence that inspire 
confidence. To contribute to this debate, the African Population and Health Research Center 
(APHRC) participated in a study that uses randomized control trials (RTC) to assess whether 
Reading to Learn (RtL), an instructional model pioneered in Australia by David Rose and used 
among the indigenous and other marginalized learners, could improve learning in low-achieving 
districts of Kenya and Uganda. To validate the replicability of this programme, which has several 
components, the implementing agency in East Africa, the Aga Khan Foundation and APHRC 
agreed to apply RCT. This article is not about the results alone but also about some of the lessons 
learned using RCT to evaluate the RtL programme. The rest of the article provides the background 
to the project and some of the findings and concludes with insights on the lessons learned. 


EAQEL—THE INTERVENTION 


East Africa Quality Education and Learning (EAQEL) is a research and development initiative 
of the Aga Khan Foundation that aims to demonstrate a model for improved learning outcomes 
in reading and numeracy of early primary grades (1-3) in two districts (Kwale and Kinango) in 
Kenya and two (Amolatar and Dokolo) in Uganda. The initiative was implemented over a period 
of 16 months. The project tested an instructional approach based on David Rose’s scaffolding 
model (RtL). RtL is a systematic approach to the teaching of reading with subsequent impact on 
numeracy. ! 

The project design includes three components: teacher preparedness and practice, school 
leadership, and classroom learning environments. These components were embedded into two 
separate but mutually inclusive modules—the “core model” and “core model plus.” The core 
model involved early grade teachers being trained on the instructional approach (EAQEL), which 
is child centered, is systematic, and focuses on social interaction. In addition, schools were 
supported to improve teachers’ and pupils’ access to and use of appropriate teaching and learning 
materials. Project technical staff worked with head teachers, key teachers, and district education 
staff from decentralized teacher support resource institutions to train teachers and provide in-class 
mentoring support. The core model plus included all of the aspects of the core model and had a 
parental involvement component. The aim was to encourage literacy by establishing minilibraries 
and encouraging parents to borrow books, read, and tell stories to their children. 

The goal of the impact evaluation was to assess the effectiveness of the EAQEL intervention. 
The objectives in undertaking this impact evaluation of EAQEL were as follows: (a) to determine 
whether the intervention leads to improved learning outcomes in numeracy and reading among 
children enrolled in Primary Grades 1, 2, and 3; (b) to determine if there is a critical difference 
in the learning outcomes of children enrolled in Grades 1, 2, and 3 attributable to the two 
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different treatment models (core model and core model plus) as proposed by AKF; and (c) to 
determine the key contributing factors that lead to improvements, if any, in numeracy and literacy 
in grades (1, 2, and 3). These factors may include, but are not limited to, the following: teachers’ 
effective implementation of the RtL approach; availability and use of instructional materials, 
in-classroom functioning libraries, head teacher active support, the presence and effectiveness 
(how engaged/involved and influential) of School Management Committees, the level of priority 
given to lower grades in the allocation of school resources (there may not be an effective School 
Management Committees but the school head prioritizes early grade, and vice versa), uptake in 
parents borrowing books and using them with their children, proximity of a functioning library, 
parental support for attendance, class size, and family literacy and education levels, among 
others. , 


THE CONTEXT 


Uganda and Kenya are two East African countries where political goodwill and international 
support have led to the Universal Primary Education and Free Primary Education policies in 1997 
and 2003, respectively. These policies have led to remarkable growth in enrollment because they 
removed the direct tuition fee that had been a major barrier to access for many children. However, 
this increase in access has potential negative effects on the quality of learning. Specifically, there 
is growing concern that millions of children spend many days in school but learn very little. For 
example, in Uganda, learners in Grade 3 scored 16.81% on a written literacy test and those in 
Kenya scored 48.35% in a similar test (Oketch et al., 2010). 


PROFILES OF STUDY DISTRICTS—KENYA 


Kwale District is an administrative district in the Coast Province of Kenya. In 2007, the larger 
Kwale district was split to form the current Kwale and Kinango districts. According to the 2009 
census, the population sizes of the Kwale and Kinango districts were 151,978 and 209,560, 
respectively. Kinango and Kwale were the worst-performing districts in the country in 2007 end 
of primary examinations. Kinango has one of the highest levels of poverty in the country, with 
half of the population living below the poverty line and two thirds considered food poor (Kenya 
National Bureau of Statistiscs, 2010). Among the challenges in the district is low community 
participation among parents in their children’s education due to poor attitudes toward education 
and also the misconception that with the FPE program parents have no responsibilities for their 
children’s education. 


PROFILES OF STUDY DISTRICTS—UGANDA 


Amolatar and Dokolo are two districts in the northern region of Uganda, which for about 20 years 
has experienced civil war due to the disgraced Lord’s Resistance Army of Kony. With the end of 
the civil war, there have been efforts by the government and a number of international partners 
to provide an effective education. AKF has been active in this regard. Literacy rates in Uganda, 
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according to the National Assessment (National Assessment of Progress in Education, 2007), 
were 45.5% at Primary Grade 3 and 49.6% at Primary Grade 6. Thus, students in Grade 6 are 
only 4 percentage points more likely to be literate. So students learn effectively nothing between 
Grades 3 and 6 if they are not literate by Grade 3. Numeracy competencies were rated at 44.8% 
at Primary Grade 3 and 41.4% at Primary Grade 6. The National Assessment of Progress in 
Education report further noted that literacy and numeracy rates of these two districts were below 
the national average. The EAQEL baseline analysis undertaken by APHRC found written literacy 
to be appallingly low at 1.63% and 5.08% for Grades | of 2010 and 2009, respectively, for the 
two districts (Oketch et al., 2010). 


EDUCATION IMPACT EVALUATION STUDIES IN AFRICA 


Recently, there has been growing interest in impact evaluation in education in the region to 
determine how best to improve educational outcomes. This has led to the prominence of RCT 
studies in the region, although overall, they are still few. Further, some object to their application in 
education on the basis of ethical concerns and their cost in setup and implementation. However, 
the proponents of RCT note that they save money in the long run because they provide clear 
treatment effects, whether positive or zero, and in so doing result in more efficient future resource 
allocation. Furthermore, because it is never known whether the treatment is helpful, harmful, or 
makes no difference, the denial of treatment through randomization is considered an ethically 
neutral scenario (Hutchison & Styles, 2010). Generally, education impact evaluation studies in 
Africa are quite few and are mainly concentrated in a handful of countries, Kenya included. 
Examples from Kenya include the flip chart study by Glewwe, Kremer, Moulin, and Zitzewitz 
(2000); merit scholarship program for adolescent girls by Kremer, Miguel, Thornton, and Ozier 
(2005); Vermeersch and Kremer (2005) on the effects of subsidized school meals on school 
participation, educational achievement and school finance; the study on teacher incentives based 
on students’ scores by Glewwe, Ilias, and Kremer (2003); Glewwe, Kremer, and Moulin (2009) 
on the impact of text books on test scores; and the study on the effect of deworming school 
children on school attendance (Miguel & Kremer, 2004). Most of these studies were concentrated 
in the districts of Busia and Teso in western Kenya where the Abdul Latiff Jameel Poverty Action 
Lab has one of its labs. 

The study on flip charts by (Glewwe et al., 2000) used retrospective and prospective analyses 
of flip chart provision to assess the effect of flip charts on student scores in rural schools. The 
retrospective analyses used data from 100 randomly selected schools involved in a separate project 
that provided textbooks and grants, prior to 1998. Although the retrospective analysis showed an 
effect of up to 20%, after controlling for other learning inputs, the prospective analyses concluded 
that there was no effect. The study argues that the observed effect in the retrospective analysis 
could have been due to selection bias. 

Another study on scholarship targeted adolescent girls who were in school and in Grade 5 
or 6 in January 2001 (Kremer et al., 2005). A total of 127 schools were randomly assigned, 
using random numbers, to either the scholarship or comparison group. End-of-year district mock 
exams were used to award scholarships to the top 15% of the girls in Grade 6. Although there 
was no significant program impact in Teso district, girls in Busia district showed large gains 
(0.22—).27 standard deviations), and the gains were sustained up to | year after qualifying for a 
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scholarship. Program externalities were also observed with boys recording sizeable average test 
gains, and both teacher and student attendance increased in treatment schools. The estimation 
strategy involved the use of a nonparametric locally weighted regression technique following an 
education production framework. In addition, the study used regression discontinuity to explore 
within-school impacts of the program in 2002—that is, a comparison of the 2002 outcomes of 
girls who barely won the scholarship to girls who barely lost out—in order to estimate the impact 
of winning. 

The study on preprimary school meals targeted 50 preprimary schools that were randomly 
divided into treatment and comparison groups of 25 schools each (Vermeersch & Kremer, 2005). 
This followed the stratification of schools by geographic location and participation in other 
projects sponsored by the implementing NGO. The 50 schools were grouped into pairs with 
similar preprogram characteristics, and a coin was tossed to select the school to be in the 
treatment group. Baseline data were collected from children 4 to 6 years of age who were living 
within 4 km of a preprimary school. Using a probit model with random effects to estimate the 
probability of a child being present in school, the results showed that school participation was 
30% higher in the treatment group than in the comparison group. The study also found that in 
treatment schools, class sizes increased despite a rise in school fees. The comparison schools 
reduced their fees. 

Glewwe et al. (2003) examined teacher incentives and their effect on students’ scores. In 
this teacher incentive study, 50 schools were selected from a group of 100 schools that were 
considered by the Ministry of Education to be in need of assistance. On average, these schools 
performed more poorly in examinations than other schools in the area (Busia and Teso districts). 
Schools were numbered alphabetically and the odd numbered schools were chosen to participate 
in the teacher incentive program. Teachers of Grades 4 to 8 participated in the study—the 
incentive was 21% to 43% of their monthly salary award at the end of the year based on 
the best-performing school and/or best-improved school in the district mock exams. The study 
examined the differences in test scores between the treatment and comparison schools using 
a random effect regression framework that allowed for the possibility that scores of students 
in the same grade and same school might be correlated due to unobserved characteristics of 
teachers and headmasters. The baseline scores were based on the district examination scores 
of 1996 (as there were no district exams results available in 1997). The program years were 
1998 and 1999, whereas the postprogram year considered in the analysis was 2000. The Kenya 
Certificate of Primary Examination scores were also used to independently evaluate the impact of 
the incentive. The study utilized the difference in test scores between treatment and comparison 
schools, and the difference-in-difference estimator of the effect of the program. Students in the 
incentive schools had higher test scores during the program period, due to short-run test scores 
effect. There was no teacher effort aimed at increasing long-run learning. The study also found that 
teacher attendance did not improve, homework assignment did not increase, and pedagogy did not 
change. 

In the textbook study reported by (Glewwe et al., 2009), 25 schools were randomly selected 
from a group of 100 schools that the Ministry of Education district office considered to be 
in particular need of assistance. Schools were listed alphabetically based on their geographical 
location. Every fourth school beginning with the first one in a list was selected to be included in the 
schools to receive treatment/text books (Group 1) at the beginning of 1996. Every fourth school 
beginning with the second, third, and fourth were assigned to Groups 2, 3, and 4, respectively. 
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Group 3 received educational improvement grants at the beginning of 1997, Group 3 at the 
beginning of 1998, and Group 4 at the beginning of 1999. The study used a random effects 
estimation procedure and then presented a levels estimator based on comparing posttest scores 
across treatment and comparison schools. Differences between pre- and posttest scores were 
also compared. The study assumed the effect of text books to be 0.20, 0.13, and 0.07 standard 
deviations for levels, difference, and subject-based estimators. Results show that there was no 
increase in average test scores, except for the bright learners (the top two quintiles of initial 
academic achievement). The intervention also increased the probability of students transiting to 
secondary school. 

These studies are useful in informing EAQEL impact evaluations in a number of ways. 
First, there is no body of knowledge that has emphasized early grades evaluating literacy and 
numeracy in Kenya and Uganda. Second, the design and interpretation of EAQEL is consistent 
with or superior to the prevailing practices in other RCT’s that have been conducted in the 
region. 


METHODS 


This study attempted to answer the following research questions: 


1. Are children in lower primary grades (1, 2, and 3) able to read and do mathematics 
calculations more proficiently as a result of the RtL/scaffolding approach? 

2. Are there differences in proficiency for children who have been exposed to parental in- 
volvement in the RtL approach (core model plus) compared to those exposed to the 
RtL approach with no parental involvement (core model), and compared to control 
schools? 

3. What are the key contributing factors to these improvements in numeracy and literacy in 
Grades 1, 2, and 3? 


Design 


RCT impact evaluations attempt to answer the following question: What would have been the 
outcome if the participants did not receive treatment (Ravallion, 2001). In most impact evaluation 
studies, a comparison group is used as a counterfactual to establish what the outcome would 
have been. In randomization, the participants are randomly assigned to treatment and comparison 
groups. The underlying principle is that any observed average difference in outcome can only 
be as a result of the program or intervention (Duflo, Rachel, & Kremer, 2006). The impact 
evaluation in this report used RCT methods. The benefit of an RCT design is the simplicity in 
result interpretation and the clear isolation of the intervention’s impact through the control group 
counterfactual while avoiding selection bias problems that exist in other evaluation designs. In 
addition to the quantitative approach of RCT, focus group discussions (FGDs) were conducted 
with parents to provide explanations for the observed results. This was restricted to core model plus 
in order to highlight mechanisms by which parental involvement might influence the outcomes 
of the intervention. 
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Sampling Procedures 
Selection of Schools 


Preexisting AKF clusters of schools in the case of Kenya and subcounties in the case of Uganda 
were randomly assigned to either the control or treatment group of the EAQEL intervention. The 
subcounties in Uganda, then, are effectively equivalent to Kenyan clusters, and we refer to both 
using the cluster terminology for the remainder of the study. By this design, all schools residing 
within treatment clusters received the treatment, and all schools within control clusters did not. 
Through clustering, contamination that may have resulted if both treatment and control schools 
were in the same cluster was minimized. 

Figure | shows the study’s design. There were a total of 41 clusters in the study, with 31 
in Kenya and 10 in Uganda. Kenya’s clusters fall within two districts, Kwale and Kinango. 
The district of Kinango was further subdivided into clusters that did or did not participate in the 
Kenya School Improvement Program (KENSIP) intervention. KENSIP was an earlier intervention 
undertaken by AKF whose effect needed to be isolated from the effect of EAQEL. Uganda’s 
clusters also fall within two districts, Amolatar and Dokolo. The final randomization occurred 
within five strata (defined by three districts, plus one district divided between KENSIP and non 
KENSIP). Of the 41 clusters, 19 received the treatment (either core or core plus, depending 


Kenya and 
Uganda (41/229) 


Kenya (31/120) Uganda (10/109) 


Kwale (17/48) Kinango (14/72) Amolatar (5/49) Dokolo (5/60) 


Treatment Core Non KENSIP Treatment Core Treatment Core+ 
KENSIP (6/36 


Treatment Core+ Treatment Core 
Control (8/22) (3/20) (3/19) Control (3/25) Control (3/32) 


Control (3/16) Control (5/17) 


FIGURE 1 Sampling frame for Kenya and Uganda. Note. The first number in parentheses is the number of Aga Khan 
Foundation clusters in Kenya, or subcounties in Uganda (i.e., the unit of randomized assignment). The second number in 
parentheses is the number of schools in all clusters/subcountries. KENSIP = Kenya School Improvement Program. 


\ 
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on the district) and 22 were in the control group. However, one school in Amolatar and one in 
Dokolo were randomly assigned to a control cluster but were later selected to be “model treatment 
schools” by AKF (a classic instance of experimental crossover between treatment and control 
conditions). 


Selection of Pupils 


The design of the study was such that it was not necessary (or financially feasible) to assess 
all pupils in each grade in the sampled schools. Therefore, a random sample of 20 pupils was 
selected in each grade. The random sampling was done by first grouping pupils by sex; and then 
selecting each sex based on their proportion in the class. Based on the baseline I experience, 
the sample was increased to 25 pupils for the 2010 Grade | in our baseline II to allow for any 
possible attrition due to absenteeism and school transfers. The same pupils were followed at the 
endline survey that took place between June and July 2011. During the endline survey, pupil 
absenteeism presented a sample attrition problem. To address the attrition problem at endline, the 
pupils who were lost to follow-up were randomly replaced, taking sex into consideration. This did 
not pose any methodological threat to the study because the intervention was administered at class 
level, 


Selection of Parents 


An initial sample of 180 parents was targeted for the focus group discussion during the endline. 
A total of 106 parents turned up for the participation of the actual FGD. The selection of parents 
was first done by randomly selecting 10% of the schools 1n the core model plus districts. The 
selected schools were assigned to be either a male or female FGD. Then, 15 pupils in each of 
the sampled schools were randomly selected and provided with letters inviting their parents to 
participate in the FGD. Among the details in the letter to the parents were the venue, time and 
whether it was the father or the mother who was invited. 

In total, 12 FGDs were conducted, five in Amolatar (three treatments and two controls) and 
seven in Kinango (four treatments and three controls). The FGDs were held separately for men 
and women, except in one school in Kenya where both male and female parents participated in 
the same FGD. 


Study Tools? 


To undertake the impact evaluation, several tools were developed. They included the following: 


Pupil assessment tools: one for each measure of Literacy and one for Numeracy 
Teacher characteristics questionnaire 

. Classroom observation checklist 

School characteristics questionnaire 


Pee 


*Please write to info@aphrc.org for more details on the tools. 
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5. Household characteristics questionnaire 
6. Focus group discussion study protocol 


Pupil Assessment Tools 


The EAQEL teaching approach focuses on literacy and numeracy in early grades 1, 2, and 3. 
In developing standardized assessment tests to assess the impact of EAQEL, several consultation 
meetings with key stakeholders and experts were held. The stakeholders included EAQEL im- 
plementing agency (AKF), APHRC, national assessment and curriculum experts, academics, and 
practitioners in numeracy and literacy assessment. These experts came from Kenya and Uganda 
as well as other countries. Several stages were involved in developing the assessment tools. First, 
a pool of questions was developed drawing on curriculum from both countries. For instance, in 
numeracy, the team came up with a pool of 50 test items in each of the grades. The competencies 
and skills for Grade 1 were examined and agreed upon by the team. In the case of Grades 2 and 3 
the competency and skills domains were repeated but the level of difficulty of test items required 
higher order thinking. 

Second, the pool was refined with the input of assessment experts, and from this pool the 
final test items were selected. Third, the test items were translated into Kiswahili and Lang’o, the 
languages of instruction in the Kenyan and Ugandan study sites, respectively. These languages 
are also widely spoken in the catchments area of these schools. 

There was one test tool for numeracy and another two test tools for each of the written and 
oral literacy for all the three grades, with students in higher grades taking all of the lower grade 
questions along with the questions appropriate for their grade. The rationale for having one 
assessment tool covering the three grades was to permit determination of how pupils in higher 
grades scored on items for lower grades (e.g., to determine how competent Grade 2 pupils would 
be on Grade | items in both numeracy and literacy). 


School and Household Characteristics Questionnaire 


Other instruments developed by APHRC and agreed on by both partners included ques- 
tionnaires to gather information on the schools, teacher’s characteristics, and household char- 
acteristics. These were adapted from ongoing APHRC research work that collects similar 
information. 


Scoring 


Scoring of the literacy and numeracy assessment test was done for each grade using the sum 
of the items that grade was supposed to attempt as the denominator and expressing a student 
score as a percentage. For example, a Grade | pupil who correctly answered all the 15 Grade 
1 numeracy items scored 100%, whereas a pupil in Grade 2 who correctly answered all the 
30 numeracy items for Grade 2 (i.e., 15 items from Grade | and another 15 at Grade 2 level) 
scored 100% (see Table 1). For the purpose of computing the impact of EAQEL using Difference 
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TABLE 1 


Number of Test Items and Total Score per Grade 
ns 


Assessment Grade Items* Total Score 
eo SS 
Oral literacy l 52 76 

2 83 120 

3 94 155 
Written literacy l 38 50 

2 a3 71 

3 74 144 
Numeracy | 15 20 

2) 30 48 

3 45 75 


a 


‘For example if the pupil was asked to read five letters, then these are five separate items. 


in Difference (DID), pupils’ endline scores were computed based on the test items they were 
supposed to respond to at the baseline. 


Description of the Sample 


This study was undertaken in two districts in Kenya (Kinango and Kwale) and two in Uganda 
(Amolatar and Dokolo). As mentioned earlier, these districts were chosen by AKF for EAQEL 
intervention because they have consistently performed poorly in national examinations. On the 
basis of our sample design described in the previous sections, this study included a total of 229 
schools distributed as shown in Table 2. 


Baseline Sample 


Baseline was carried out in two phases. The first phase was conducted in July and August 
2009 and targeted 9,160 pupils in both Grades | and 2. The second phase was carried out in 
February and March 2010 for incoming Grade | and targeted 5,725 pupils. This was in line with 


TABLE 2 
Distribution of Schools by District 
Control Treatment 
District No, % No. % 
Kinango 33 45.83 39 54.17 
Kwale ep 45.83 26 54.17 
Amolatar 24 48.98 2) $1.02 
Dokolo 31 51.67 29 48.33 


Total 110 48.03 US) Soy 
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the intervention period and aimed to capture all three grades (1, 2, and 3) in the impact evaluation. 
However, in the actual baseline test, the number (14,404) of pupils assessed was less than the 
target (14,885). The reasons for the difference between the target and actual were (a) some classes 
had fewer pupils below the target sample size of 20 pupils in 2009 and 25 in 2010 and (b) during 
the testing time, a few pupils disappeared from the test venues and some were also absent during 
call backs. 


Endline Sample 


During the endline (follow-up), a total of 13,944 pupils were captured. This consisted of 9,397 
pupils followed from the baseline (67.4% of the baseline sample). The rest were new pupils 
sampled to replace those lost mainly due to absenteeism. Table 3 shows the proportion of pupils 
who were assessed for each of the assessment tests, both at baseline and endline. Table 3 also 
shows the follow-up group and the number of pupils who could not be traced. 

During the endline survey, we targeted teachers in treatment schools who were captured at the 
baseline irrespective of the grade they were teaching in 2011 as well as those currently teaching 
Grades | to 3. In total, 445 teachers were interviewed and the distribution of the number of 
teachers who were interviewed is as shown in Table 4. 


Attrition 


Even when baseline attributes are balanced across treatment and control groups, some students 
cannot be located at the endline. The main concern is that attrition is nonrandom (1.e., lower 
achieving students leave the sample) and that patterns of attrition differ across treatment and 
control groups. For example, one could imagine that a treatment could prevent student drop-out 
(or attrition), thus altering the relative composition of students in treated schools versus control 
schools at the follow-up. These differences in endline achievement, introduced by attrition, could 
be mistaken for the impact of the treatment. 

Table 5 examines treatment and control group attrition. Panel A shows that, overall, about 
34% of the baseline students in the treatment group could not be located at the endline. However, 
about 38% of the control group could not be located. The difference between the likelihood 
of attrition across the two groups is not statistically significant. This provides a measure of 
confidence that differential attrition did not create further imbalance in the treatment and control 
groups. 

Panel B in Table 5 compares the baseline scores of “attritors” from the treatment group and 
those from the control group. Note that, on average, treatment group attritors are lower achieving 
than those in control groups (though not statistically significant). Also note that the difference 
is very similar to the difference in baseline test scores among all students. Thus, the attrition 
does not appear to have introduced further imbalance in baseline test scores across treatment and 
control groups. In conclusion, although levels of attrition are high—and not surprisingly so, given 
the context—differential attrition does not seem to be an important threat to the internal validity 
of the estimated treatment effects. 
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TABLE 4 
Distribution of Teachers Interviewed 
District Teachers (n) % 
Kinango 135 30.34 
Kwale 104 23.37 
Amolatar 84 18.88 
Dokolo 122 27.42 
Total 445 100.00 





Baseline Balance Across Treatment and Control Groups 


At the baseline, there was small difference in test scores (Oketch et al., 2010) in favour of 
the control group in Kenya, particularly in numeracy. This has implications for the analysis of 
the end results because ideal randomization assignment implies that baseline test scores should 
be similar, on average, across treatment and control groups. Overall, Table 6 indicates that 
students attending treated schools at baseline have modestly lower baseline test scores, although 
the differences are only statistically significant in the case of the overall numeracy test (pooling 
Kenyan and Ugandan samples) and in the numeracy test only in the Kenyan sample. In oral literacy, 
for example, the differences are approximately 12% of a standard deviation and statistically 
insignificant. 

To address the baseline imbalance in test scores, the analysis at the endline used DID 
technique, which is a straightforward and clear way of assessing the treatment effect of the 
intervention. 


RESULTS OF THE RCT IMPACT EVALUATION 
Treatment Effects on Numeracy 
Table 7 shows the pooled DID data for Kenya and Uganda. The results indicate that there is 


no treatment effect on numeracy. At the country level results shown in Table 8, one of the key 
highlights is that in Grade 1, 2010 shows a DID of —3.92 percentage points, which indicates 








TABLE 5 
Attrition 
Treatment Control Difference 
Panel A: Proportions 
Proportion of baseline cohort not in endline 0.342 0.376 =,035 
Panel B: Scores of those in baseline cohort not in endline 
Numeracy score —0.161 —0.023 =.137 
Written literacy score —0.141 —0.002 =.139 
Oral literacy score —=Orl13 0.014 —.127 


er ——————————————————————————————————————— 


Note. Treatment and control group as originally defined by African Population and Health Research Center. 
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TABLE 6 


Baseline Test Scores 
pe ct te Yk i eee ee 





Treatment Control Difference 

Numeracy score —0.078 0.076 =153* 
Kenya —0.122 0.126 —.248** 
Uganda —0.030 0.027 —.057 
Written literacy score —0.076 0.075 —.151 
Kenya —0.089 0.093 —.182 
Uganda —0.062 0.057 —.120 
Oral literacy score —0.061 0.060 —.121 
Kenya —0.060 0.062 —.123 
Uganda : —0.063 0.058 —.120 





Note. Mean of standardized test scores. Treatment and control group as originally assigned by African Population 
and Health Research Center. 
*Significant at 10%. **Significant at 5%. 


that schools in the control group performed better in Kenya. In Uganda, Grade 1 of 2010 
shows a positive DID of 6.45 percentage points in favour of the treatment group. The statistical 
significance shown in the two cohorts is possibly noise introduced by the low number of clusters 
and, therefore, the pooled data indicating no treatment effect on numeracy in both countries is more 
reliable. 

Table 9 presents the impact of EAQEL at the district level. These results suggest that these 
effects are similar across the two districts in Kenya, but due to the fewer number of clusters 
in each district, we cannot precisely assess their statistical significance. The point estimates for 
Dokolo are particularly positive for the Grade | 2010 cohort. Similarly, that of Kinango is large 
compared to the others, but due to the small number of clusters in each district, these estimates 
are not statistically significant. Therefore, the differences in the DID point estimates between 
the districts or differences of the point estimates from zero should be taken not as statistically 
significant, but informational. 


Pupil Abilities at the Baseline 


The EAQEL intervention was designed to have a greater impact among those pupils who 
were not able to do numeracy and literacy on their own. To examine the impact of EAQEL 


TABLE 7 
Difference in Difference (DID) in the Numeracy Assessment Both Countries 





Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 


Treatment (t;—to) 9.44 18.81 22.37 
Control (t;—to) 9.39 17.63 19.70 
DID 0.04 ~ 1.18 2.68 





Note. t; is endline, to is baseline. 
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TABLE 8 


Difference in Difference (DID) in the Numeracy Assessment by Country 
ere Rn a eee en eS) 





Country Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 

Kenya Treatment (t}—to) ioe 16.12 28.16 
Control (t;—to) 1.25 14.75 25.38 
DID —3.92** 137), 2.78 

Uganda Treatment (t)—to) 13.08 23.35 Lae 
Control (t;—to) 6.63 22.31 11.50 
DID 6.45** 1.04 I 75) 





Note. t; is endline, to is baseline. 
Sip = 05: 


on the learning outcomes of low- and high-performing pupils within each district, we placed 
pupils into three categories according to their performance at baseline: those who scored 
below | standard deviation from the mean (the lowest scoring students), those who scored 
within | standard deviation below and above the mean (the middle scoring students), and 
those who scored more than | standard deviation above the mean (the highest scoring 
students). 

Table 10 presents the results based on standard deviation categorization for Kenya and Uganda. 
It is notable that there was no treatment effect on numeracy across the cohorts among pupils who 
scored higher marks at the baseline. In Kenya, Grade 1, 2010 cohort pupils scoring | standard 
deviation below the mean gained statistically significantly less than the control group. In Uganda, 
Grade 2, 2009 cohort, a differential positive impact was seen among those ranked | standard 
deviation below the mean. 





TABLE 9 
The Difference in Difference (DID) Based on Numeracy in Kenya 
District Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Kinango Treatment (t;—to) 132 16.89 30.42 
Control (t;—to) 10.80 13.92 23.19 
DID —3.48 2.97 7.23 
Kwale Treatment (t;—to) 7.34 15.02 25.06 
Control (t;—to) 11.86 15.99 28.69 
DID —4,52 —0.96 —3.62 
Amolatar Treatment (t;—to) 18.58 27.03 18.83 
Control (t;—to) 13.63 233) 16.17 
DID 4.95 5.69 2.66 
Dokolo Treatment (t;—to) 9.56 20.80 8.44 
Control (t;-to) 1.27 23.00 8.21 
DID 8.29 —2.20 0.23 


i a a le i a a I Se ee 
Note. Due to fewer numbers of clusters at district level of analysis, statistical significance cannot be determined for 
the point estimate differences shown in the table. 
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TABLE 10 


Learning Outcomes of Low- and High-Performing Pupils Based on the Numeracy Scores 
i i 8 a 


Grade/District Group <-1SD Within —1 & 1 SD > +1SD 
pi a leew 
Grade 1, 2010 
Kenya Treatment (t;—to) 23.70 SS —4.81 
Control (t;—tp) 29,94 9.71 —4.57 
DID —6.25** —3.97** —0.24 
Uganda Treatment (t|—to) 29.07 14.86 —15.41 
Control (t;—to) 32.90 10.19 —20.30 
DID —3.84 4.67 4.88 
Grade 1, 2009 
Kenya Treatment (t;—to) 32.68 14.40 —0.27 
Control (t;—to) 37.65 14.19 —0.19 
DID —4.98 0.21 —0.18 
Uganda Treatment (t;—to) 43.22 Das 1.61 
Control (t;—to) 41.85 25.44 —1.38 
DID 1.37 0.30 2.98 
Grade 2, 2009 
Kenya Treatment (t;—to) 45.32 27.92 9.85 
Control (t;—to) 46.22 26.73 10.05 
DID —0.90 1.19 —0.20 
Uganda Treatment (t;—to) Dahil 13.65 —0.40 
Control (t;—to) 18.63 13.42 —1.21 
DID 6.48** 0.23 0.81 





**y < 05. 


Summary of the Numeracy Assessment Results 


The DID presented in Tables 9 and 10 indicate that EAQEL had no overall effect in improving 
numeracy when the countries are combined or in each country separately. As noted earlier, 
this is a true reflection of absence of treatment effect in numeracy achievement as the DID 
takes into account any differences that may exist between treatment and control groups at the 
baseline. 

Although the district level results show an unbiased point estimate difference, their significance 
cannot be determined due to fewer numbers of clusters. 





TABLE 11 
Difference in Difference (DID) in the Oral Literacy Assessment Both Countries 
Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Treatment (t;-to) Veo 21.19 18.51 
Control (t -to) 15.69 19.53 16.02 
DID 1.81 1.66 2.49 





Note. t, is endline, tp is baseline. 
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TABLE 12 


Difference in Difference (DID) Between Treatment and Control in Oral Literacy by Country 
re ene a ee eee 8 





Country Group Grade |, 2010 Grade 1, 2009 Grade 2, 2009 
Kenya Treatment (t}—to) 18.60 21.88 19.03 
Control (t|—to) 18.69 21.81 18.64 
DID —0.09 0.07 0.39 
Uganda Treatment (t;—to) 15.64 20.16 17.65 
Control (t;—-to) 11.19 15.62 12.20 
DID 4.45** 4.54** SASS 





Note. t; is endline, tg is baseline. 
=p = 05: 


Treatment Effects on Oral Literacy 


Tables 11 and 12 present country specific treatment effects of the EAQEL intervention based 
on the oral literacy scores. In the table, the treatment and control row entries show the mean 
difference between endline and baseline (score increases) for each of the groups, whereas the 
DID row presents the percentage point difference in difference between the treatment and control 
groups (i.e., the treatment effect). 

The DID results in Uganda are positive and significant across all the three cohorts, whereas 
none is significant in Kenya. These results indicate that EAQEL/RtL had a positive treatment 
effect on oral literacy in Uganda but not Kenya. 

Table 13 presents a comparison of the treatment and control schools by district, in 
Kenya and Uganda, respectively—although, as noted earlier, district level analysis in terms 


TABLE 13 
The Difference in Difference (DID) Based on Oral Literacy 
District Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Kinango Treatment (t;—to) 17.86 24.02 23.02 
Control (t;-to) 17.62 23.32 21.19 
DID 0.24 0.70 1.83 
Kwale Treatment (t;—to) 19.63 18.55 13.53 
Control (t;—to) 20.19 19.58 14.87 
DID —0.57 —1.02 —1.33 
Amolatar Treatment (t;—to) 20.40 25.14 21.49 
Control (t;-to) ADaL9. 20.08 12.08 
DID 5.21 5.06 9.41 
Dokolo Treatment (t;—to) 12.65 16.74 14.90 
Control (t;—to) 8.06 1250 12.28 
DID 4.59 4.23 2.63 





Note. Due to fewer numbers of clusters at district level of analysis, statistical significance cannot be determined for 
the point estimate differences shown in the table. 
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TABLE 14 


Learning Outcomes of Low- and High-Performing Pupils Based on the Oral Literacy Scores 
ee ree 





Grade/District Group <-ISD Within —1 & 1 SD > +1 SD 
Grade 1, 2010 
Kenya Treatment (t;—to) 29.10 17.97 10.68 
Control (t;—to) 29.22 18.13 10.20 
DID —0.13 —0.16 0.48 
Uganda Treatment (t;—to) Slee 15.30 1.99 
Control (t;—to) 28.12 11.10 —2,39 
DID 3.61 4.20 4.39 
Grade 1, 2009 
Kenya Treatment (t;—to) 36.55 22.39 7.40 
Control (t;—to) 35.89 22.69 8.86 
DID 0.67 —0.30 —1.46 
Uganda Treatment (t;—to) 41.50 Nene 35 
Control (t;—-to) 38.02 14.39 2.95 
DID 3.47 B32 4.40 
Grade 2, 2009 
Kenya Treatment (t;—to) 30.12 21.24 3.26 
Control t;—to) 33.33 20.41 3.46 
DID —3.21 0.84 —0.21 
Uganda Treatment (t;—to) 34.72 16.34 3.87 
Control (t;—to) 26.11 12.91 0.00 
DID 8.61 3.43** 3.87 
PDO): 


of significance levels has to be interpreted cautiously due to fewer numbers of clus- 
ters. Consistent with country-level results, there is no treatment effect in both districts 
in Kenya. 

In Uganda, the core model plus was implemented in Amolatar district and the core model in 
Dokolo. EAQEL shows a positive impact on oral literacy across the three grades in both Amolatar 
and Dokolo. In Amolatar, an impact of 5 percentage points for the cohorts of both Grades 1 of 
2010 and 2009 is observed. The most striking DID result is the 9 percentage points for 2009 
cohort of Grade 2 in oral literacy. In Dokolo, the highest DID was observed in the 2010 cohort of 
Grade | (4.6 percentage points) and the least with the 2009 cohort of Grade 2 (about 3 percentage 
points). This suggests that, based on point estimates alone, the core model plus had a greater 
impact than the core model in Uganda. 


Pupil Abilities at the Baseline 


Table 14 presents results by categorization of pupils’ performance at baseline by how far their 
score was from the mean score below | standard deviation, within | standard deviation, and above 
1 standard deviation of the mean. In Kenya, the gains are large, but there is no EAQEL treatment 
effect on oral literacy across the initial performance categories. In Uganda, an average performing 
pupil (within | standard deviation) consistently benefited more from the intervention. Contrary 
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TABLE 15 
Difference in Difference (DID) in the Written Literacy Assessment Both Countries 
Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Treatment (t)—to) 2245 29.43 D1al9 
Control (t)—to) 19.74 28.84 24.69 
DID 2.40 0:59 3.10 





Note. t; is endline, to is baseline. 


to the expectations of the intervention, high achievers in Uganda also show a rather impressive 
treatment effect across all grades. Nonetheless in Amolatar there is impressive treatment effect 
of about 7 percentage points among the low-performing pupils in Grade 2 of 2009 (results not 
presented). 


Written Literacy Treatment Effects 


Tables 15 and 16 present country-specific treatment effects (DID) of the EAQEL intervention 
based on the Written Literacy. The Written Literacy results show a positive treatment effect in 
Uganda across the three cohorts, whereas in Kenya, there is none. 

Table 17 presents a comparison of the treatment and control schools by district in Kenya and 
Uganda, respectively. The gain scores (t;—to) for the treatment and control groups are highly 
significant. Larger DID were observed among pupils in Uganda compared to those in Kenya, 
though pupils in Uganda had much lower baseline mean scores. 

In Kinango, where the core model plus was implemented, results show positive DID point 
difference for the Grade 2 of 2009 cohort. In Kwale, where the core model was implemented, the 
control Grade 1 of 2009 cohort, which is control group, shows a positive DID point difference 
of 4 percentage points. In Uganda the DID point differences are larger than those observed in 
Kenya, but the statistical significance cannot be determined due to fewer cluster numbers at the 
district level. 





TABLE 16 
Difference in Difference DID Between Treatment and Control in Written Literacy by Country 
Country Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Kenya Treatment (t}—to) 27.29 31.95 27.89 
Control (t;—to) 27.49 34.70 25.69 
DID —0.20 —2.75 Del 
Uganda Treatment (t;—to) 12.83 25.48 27.90 
Control (t;—to) 8.10 19.24 23.27, 
DID 4.73** 6.24** 4.63** 





** p05: 
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TABLE 17 
The Difference in Difference (DID) Based on Written Literacy by District, Kenya 
District Group Grade 1, 2010 Grade 1, 2009 Grade 2, 2009 
Kinango Treatment (t;—to) 24.52 33.32 30.60 
Control (t;—to) 24.57 34.98 28.01 
DID —0.05 —1.66 2.60 
Kwale Treatment (t;—to) eile all 29.88 D575 
Control (t;—to) BST 34.28 22.23 
DID —0.30 —4.4] 1352 
Amolatar Treatment (t;—to) 14.51 24.86 32.97 
Control (t;—to) 12.19 19.14 22.68 
DID 232 Sale 10.29 
Dokolo Treatment (t;—to) PETS 25.91 24.29 
Control (t;—to) 4.91 19.31 23.69 
DID 6.87 6.60 0.60 


Note. Due to fewer numbers of clusters at district level of analysis, statistical significance cannot be determined for 
the point estimate differences shown in the table. 


TABLE 18 
Learning Outcomes of Low- and High-Performing Pupils Based on theWritten Literacy Scores 


Grade/District Group <—1 SD Within —1 & 1 SD >+1SD 


Grade 1, 2010 
Kenya Treatment (t;—to) 18.57 29.37 20.07 
Control (t;—to) 22.36 29.00 22.76 
DID —3.79 0.38 —2.69 
Uganda Treatment (t;—to) — eS 23.69 
Control (t;—to) — 8.01 9.14 
DID — 356"* 14.55*** 
Grade 1, 2009 
Kenya Treatment (t;—to) 29.73 36.27 16.72 
Control (t;—to) 35.08 39.86 18.59 
DID —5.35 —3.59 —1.86 
Uganda Treatment (t;—to) — 2ae 34.16 
Control (t;—tg) — 18.32 26.05 
DID — 6.44*** 8.11 
Grade 2, 2009 
Kenya Treatment (t;—to) 37.98 30.83 7.43 
Control t;—to) 34.05 30.27 10.40 
DID 3.93 0.56 —2.97 
Uganda Treatment (t;—to) 27.37 28.67 22.67 
Control (t;—to) DOEYD, 22.85 25.53 
DID 4.85 S827 5 —2.86 


a 
Note. The missing (—) mean scores for Uganda is a result of very low mean scores for Written Literacy at the baseline 
for Grades 1 2010 and 2009 cohorts such that few pupils fell below 1 SD from the mean. 
eae re) le 
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Pupil Abilities at the Baseline 


Table 18 presents the results based on standard deviation categorisation for districts in Kenya and 
Uganda, respectively. In Kenya, there is no treatment effect on oral literacy across pupil ability 
levels in all the cohorts. 

In Uganda, the mean scores for Written Literacy at the baseline were very low for Grade | in 
2010 and 2009, such that few pupils fall below | standard deviation from the mean. EAQEL had 
a positive effect for the pupils ranked within | standard deviation of the mean at their baseline 
scores. For the pupils who at baseline were | standard deviation above the mean, the EAQEL 
intervention had a positive and large DID benefit of about 15 percentage points among the Grade 
lcohort of 2010 in Uganda. 


The Role of Program Implementation 


After implementation, but without knowing the endline test scores, AKF created a simple catego- 
rization of the treated schools based on the fidelity of program implementation (or uptake). The 
three categories—high, medium, or low—are based on 11 separate indicators of implementation 
observed by AKF personnel. 

Table 19 examines whether the treatment effects are sensitive to the degree of program im- 
plementation. In the first column, three separate coefficients indicate treatment effects among 


TABLE 19 
Treatment Effects in High-, Medium-, and Low-Implementing Schools 
Global Mean Heterogeneous Effects 
Written Oral Written Oral 


Numeracy Literacy Literacy Numeracy Literacy Literacy 


Intention to Treat x High Uptake 0.033 ONO O19" 

Intention to Treat x Medium Uptake 0.012 0.087 ONGs%" 

Intention to Treat x Low Uptake —0.064 —0.018 0.036 

intention to Treat x High Uptake x Kenya 0.029 0.097* OnG5 ae 
Intention to Treat x Medium Uptake x Kenya =0.099 —0.030 0.020 
Intention to Treat x Low Uptake x Kenya —0.200** —0.111* —0.060 
Intention to Treat x High Uptake x Uganda 0.013 OBS8as 0.300** 
Intention to Treat x Medium Uptake x Uganda 0.147 0:233* 0.340*** 
Intention to Treat x Low Uptake x Uganda 0.138 0.118 0.179* 
N 8,920 8,850 8,819 8,920 8,850 8,819 
R? 0.28 0.34 0.29 0.28 0.35 0.29 





Note. Sample of students who completed specified endline test and at least one baseline test. Endline test scores 
standardized based on cohort, country, and grade at endline. Intention to treat defined through original African Population 
and Health Research Center randomization. All regressions include controls for all three baseline tests (students who 
did not take a particular test are given a score of 0), a dummy variable for each missing baseline test score, interactions 
between cohort and country, a dummy variable for sex, and district fixed effects separately by Kensip status for all but one 
district in each country. Standard errors clustered at the unit of randomization (cluster for Kenya, subcounty for Uganda). 

*Significant at 10%. ** Significant at 5%. ***Significant at 1%. 
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schools with high, medium, or low implementation. Note that implementation does not mat- 
ter for numeracy effects, since all are small and statistically indistinguishable from zero. For 
both literacy assessments, the full-sample effects are highest among the high implementation 
category (19-22% of a standard deviation). In contrast, they are zero among the low category 
of schools. 

It is also instructive to assess treatment effects by implementation status in each country. The 
final columns of Table 19 confirm once again that in Uganda the treatment effects are uniformly 
positive for all levels of implementation. However, note that effects are relatively lower in low- 
implementing schools. In Kenya, the results are particularly striking because they now show 
small and statistically significant treatment effects (10-17% of a standard deviation) but only in 
high-implementing schools. 

In conclusion, the results suggest that implementation quality, as judged by the criteria and 
observation of AKF, is an important mediator of program effects. It even suggests that, among a 
subset of Kenyan schools, there were small positive effects. 


CONCLUSIONS AND DISCUSSION 


The following broad conclusions can be drawn from the results presented in the various tables 
on the impact evaluation of EAQEL: (a) on average, the EAQEL intervention was effective at 
raising Ugandan literacy, both written and oral; (b) it was not effective at raising Kenyan literacy; 
and (c) it was not effective at raising numeracy in either country. 

The results present an important puzzle. What can explain the generally positive findings 
in Uganda, and the much weaker findings in Kenya? The first potential explanation is that the 
baseline achievement and resource levels were lower in Ugandan schools. In this regard, the 
two districts included in the study had suffered greatly in the recent Ugandan conflict and had a 
more limited supply of instructional materials than the districts in Kenya. The provision of any 
instructional materials or libraries could more easily have an immediate effect on students who 
would have had very few materials in the absence of treatment. In addition, the EAQEL-provided 
instructional materials in Uganda were the first materials available to these schools in the local 
language. In Kenya, with better resourced classrooms and higher levels of initial achievement, 
the intervention’s effect might have been weaker. 

A second explanation is that the assessment tools showed some evidence of ceiling effects. 
The scores of Kenyan students—who had higher initial levels of achievement—were at the top 
of the measurement scale. That is, some students answered all items correctly. In this case, the 
growth of treated Kenyan students’ test scores is attenuated relative to control students. Even 
so, this explanation bears a more careful examination of the item-level test score, and the use of 
alternative statistical techniques to analyze the data. 

A third explanation comes from the potentially differential implementation in the two countries. 
This explanation was somewhat explored in Table 19, but important differences could still remain. 
Different individuals, with potentially varied ideas on adequate implementation, rated school-level 
implementation in the two countries. Also, teachers were trained in their respective countries. 
Other differences include an emphasis on including all the steps of the RtL framework in a single 
lesson in Kenya, whereas the thematic baseline curriculum in Uganda lent itself to splitting the 
steps across multiple lessons; and the differential model of funding “support tutors” resulted in 
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more follow-up with the treatment schools in Uganda than in Kenya. Although these explanations 
are speculative, they can serve as the basis for additional analysis of quantitative and qualitative 
data, as well as discussion with AKF, 


Lessons Learned 


RCT as an evaluation method is still debated and still unacceptable by some education practitioners 
and researchers, the argument being that this method borrowed from the medical field doesn’t 
address the complexity in the education field. The moral dilemma question has always been about 
treating some and leaving others out. Those opposed to RCT claim this is a major weakness 
and justifies why it is not appropriate in education. There are rejoinders, nonetheless, who argue 
that the ultimate benefit is to everyone. The dangers are minimal to everyone, too (Hutchison 
& Styles, 2010). This debate still goes on, and while implementing EAQEL, we experienced it. 
Many of the control schools felt that they should have been part of the study and some of the 
community leaders were so enthusiastic that they had already promised their local schools that 
a new programme that would benefit their children was soon to be rolled out. They wanted to 
take credit for this, even if they didn’t care what RCT was about. They didn’t in fact understand 
what RCTs are. So, when the baseline was conducted and some schools eventually selected as 
treatment and others left out as control, there was much disappointment. The extent to which this 
affects the implementation, local support of project, and its effect on results is part of the debate 
on RCT that warrants further reflection on the application of RCT in education. 

Most RCT are small in scale, and the programme implementer doubles as the evaluator. Where 
there is independent impact evaluation, as was the case in EAQEL, there can be several complex 
challenges. The first challenge is to overcome the interest of the implementing agency that there 
shouldn’t be “special” schools that must be part of the study. This is never easy, because in the 
case of EAQEL, the implementing agency had worked with some of the schools for decades and 
felt that such schools should be included as treatment schools. This brings in the challenge of 
contamination, too, because these schools tend to be located in one geographical area. It requires 
strong negotiation and agreement, and often there will be a few mistakes along the way, where 
some schools not required to be in treatment are included, or some thought to be in treatment end 
up in control. 

Implementing agencies, if not used for independent evaluation, often expect the results to be 
positive, as the practice is usually to use pre—post methods that do not truly capture programme 
effects. This is a natural expectation, particularly because the costs involved in implementing 
projects with an RCT component are very high. When the results are negative, acceptance 
is often difficult, and the questions raised often go back to wanting to say that RCT is not 
appropriate. 

The independent evaluator usually relies on the good will of the implementer through mon- 
itoring an evaluation system to offer accurate implementation information that would facilitate 
analyses of the effect of uptake on the results. Such information can be complex to gather and 
normally the independent evaluator may want to be less involved in the implementation and not 
be privy to what has actually happened during the implementation. 

Implementation where teachers are involved can be complex because different teachers and 
schools may partake to implement the project differently. Without clear documentation of school 
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processes that facilitate or undermine the project, implementation information used in the impact 
evaluation can be misleading. 

Where two or more countries are involved, comparison is inevitable, yet it can be clear that 
the practices in the different countries may lead to different results. This information is hardly 
available to the independent impact evaluator and at the stage of analysis, what may emerge 
could be wrong treatment effect because one country may have just had the right conditions that 
allow such effect to occur. An example might be a case where one country has no resources and 
the other has resources, such that any introduction of resources associated with a project may 
indicate treatment effect and lack of effect on those with resources. Project implementers often 
want greater attribution to indicate that a project works, but sometimes, in some of the projects, 
separating the component that actually led to the effect can be difficult. In the case of EAQEL, 
it is unclear if the positive literacy results observed in Uganda could be attributed to teachers or 
to resources that were introduced. Because Kenya already had these resources, it is difficult to 
separate the true effect of the composite EAQEL parts. 

Although RCT is appealing, the EAQEL study in East Africa indicates is that it is complex 
to introduce in the field of education; Banerjee and Duflo (2011) claimed that it is the “gold 
standard” to address global economic issues and understand the poor is widely appealing, but 
the complex side of this approach in education needs more debate and research. The fallouts and 
what Heyneman (2012) referred to as “monopoly of methodology” needs to be considered before 
the major donors such as DFID and USAID all devote their resources toward RCT for evidence 
gathering in education. 
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The “Developing” Achievement Gap: Colombian 
Voucher Reform 
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The achievement gap in many developing countries is defined in terms of rich/poor and public/private. 
The prevailing explanation for the “developing” achievement gap is an underfunded, inefficient, and/or 
inadequately supplied public school sector. Via an analysis of a Colombian voucher experiment, this 
article examines the extent to which income-contingent vouchers can narrow the achievement gap 
and provide a cost-effective method for increasing secondary school enrollments. Despite structural 
and implementation flaws, which diminish the program’s impact on achievement and enrollments, 
its successes strengthen the argument that implementing an income-contingent voucher program can 
help narrow the achievement gap in developing countries. To cost-effectively increase enrollments, 
however, significant modifications and expansions to the program would be necessary—as explained 
in the conclusion of this article. 


Colloquially referred to as No Child Left Behind, the full title of arguably the most comprehensive 
federal education bill in the history of the United States reads, “An Act: To close the achievement 
gap with accountability, flexibility, and choice, so that no child is left behind” (No Child Left 
Behind, 2001, p. 1425). Without any further elaboration, one would expect that nearly all education 
researchers in the United States would be able to provide an appropriate explanation for what is 
meant by the “achievement gap” in the title of this bill. A term bandied about in the field with 
great regularity, in its simplest form “the ‘achievement gap’ in education refers to the disparity 
in academic performance between groups of students” (Editorial Projects in Education, 2004, 
p. 1). Although this interpretation does allow for the achievement gap to be defined for any 
and all groups of students, the EPE goes on to offer that in the United States, “it is most often 
used to describe the troubling performance gaps between many African American and Hispanic 
students, at the lower end of the performance scale, and their non-Hispanic white peers” (p. 1). 
It is important to note, however, that this universally accepted definition of the achievement gap 
within our country’s borders is not applicable everywhere. 

In developing countries, the achievement gap is most frequently defined for four distinct—but 
not mutually exclusive—groups of students: (a) rural versus urban, (b) national (dominant) 
language versus indigenous language, (c) rich versus poor, and (d) public school versus private 
school (Bettinger, 2005; Hornberger, 2000; James, 1993; Lopez & Kuper, 2000; McEwan & 
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Carnoy, 2000; Somers, McEwan, & Willms, 2004; Uribe, Murnane, Willett, & Somers, 2006). 
Although all four of these gaps are of serious concern in developing countries throughout the 
world, this article focuses specifically on the divide between rich and poor, as well as that between 
public and private school students—which, due to the structure of the education system in many 
developing countries, are highly correlated (James, 1993; Uribe et al., 2006). 

The prevailing explanation for the achievement gap between rich and poor—and public and 
private—in the developing world stems primarily from public expenditures on education. When 
governments are unwilling or unable to sufficiently subsidize public education, the inadequate 
supply of public school spaces (in conjunction with the often low-quality public sector) creates 
a demand that is met by the private school system (James, 1993; Somers et al., 2004; Tooley & 
Dixon, 2005). The problem is often exacerbated at the secondary school level because the primary 
level is often the main focus of governmental expenditures on education. This is supported by 
the work of Estelle James, who—in a study of 50 “advanced industrial societies” and developing 
countries—found that the mean percentage of students enrolled in private secondary schools was 
nearly 50% higher in developing countries (James, 1993). Although recent trends have given 
rise to a low-cost private sector across the globe, the traditionally short supply of public school 
spaces, coupled with high prices of private schools, ultimately lead to a rich-poor enrollment 
gap in many developing countries. In addition, even for those students who have the opportunity 
and means to attend school, the differing quality in schooling options (for high- and low-income 
students) remains significant and has served to widen the achievement gap (Angrist, Bettinger, 
Bloom, King, & Kremer, 2002; King, Rawlings, Guitierrez, Pardo, & Torres, 1997; Uribe et al., 
2006; World Bank, 2011). 

When government funding and public school spaces are limited, governments and aid organi- 
zations have employed a variety of programs and initiatives to increase high-quality educational 
access for all students. Although these programs (including cash transfers, scholarships, conces- 
sionary schools, etc.) all have their merits, some countries have chosen to implement income- 
contingent voucher schemes in an effort to level the playing field. As a supply-side solution, 
income-contingent vouchers offer the promise of increased capacity (by providing private school 
vouchers to public school students—thus freeing up public school spots for those previously 
unable to attend) and increased opportunity (especially for those moving from public to private 
schools), both of which have the presumed benefit of narrowing the rich—poor gap with regard to 
enrollment and achievement (Arenas, 2004; Hsieh & Urquiola, 2003; Kremer, 2002; Zimmer & 
Toma, 2000). 

Via an in-depth examination of one such voucher experiment, el Programa de Ampliacién 
de Cobertura de la Educacién Secundaria (PACES)—a secondary school voucher program in 
Colombia—this article attempts to answer the following questions: Can income-contingent (a.k.a. 
overflow) vouchers help narrow the achievement gap in developing countries? Are educational 
vouchers an efficient (i.e., cost-effective) method for increasing secondary school enrollment? 
What lessons can be learned from the design of the Colombian voucher reform? 


PACES 


Educational privatization and school choice are two of the most regularly (and often con- 
tentiously) discussed topics in education today. It is important to note, however, that educa- 
tional privatization—although frequently opposed or advocated prima facie—is a multifaceted, 
multidimensional approach to school reform that has taken on various forms over the years. 
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For example, tuition tax credits [have been] provided to parents to partially or fully compensate them 
for private school tuition; publicly funded educational vouchers or certificates [have been] used to 
pay for private schools; and public education authorities [have contracted] with private, for-profit 
entities to operate their schools. (Belfield & Levin, 2005, p. 2) 


Although examples of each of these approaches can be found in education systems throughout 
the world—and an examination of each would surely provide tremendous insight into the effects 
of privatization policies worldwide—this article is limited to a specific evaluation on the use of 
educational vouchers for secondary school students in Colombia. 

In a simplified generalization of the voucher debate (not particular to Colombia): Critics of the 
voucher system customarily raise concerns about religion in schools, social cohesion, and educa- 
tional equity, whereas voucher proponents typically assert that vouchers provide students/families 
with greater freedom of choice and will offer a more efficient educational system via the free mar- 
ket. Unfortunately, people all too often take immutable stances on either side of the educational 
voucher debate—on primarily ideological grounds—without a full comprehension of the com- 
plexity and nuance inherent in any given voucher system. Consequently, this article provides a 
thorough examination of the finance, regulations, and support services of the Colombian voucher 
program before drawing any conclusions about its relative merits with regard to narrowing the 
achievement and/or enrollment gap for secondary school students. 

Although education policy reform debates in the United States often revolve around how 
to optimize educational opportunities for primary and secondary school students, ‘discussions 
about vouchers in developing countries center on the supply of and access to primary and 
secondary education” (Bettinger, 2005, p. 5). Subsequently, “the voucher debate in Colombia 
was not about competition; it was about how to increase the capacity of the country to promote 
and manage additional enrollment” (Bettinger, 2005, p. 5). Despite an impressive expansion of 
investment in the education sector in Colombia throughout the 1960s and early 1970s—leading to 
an increase in the average years of schooling from 2.5 to 5.5—“the growth of the education sector 
virtually stopped around 1975, mainly as a consequence of institutional problems, extremely 
centralized management, and lack of political will to make the necessary investment and changes” 
(Calderon, 1996, p. 1). Although primary school enrollment continued to increase—reaching 85% 
in 1990—secondary school enrollment suffered a different fate, with approximately one half of 
the eligible students attending secondary school (Calderon, 1996). The situation was far worse 
for students in the poorest two quintiles, as “the gap in enrollment rates between the poorest and 
richest students in Colombia was almost 35 percent” (Bettinger, 2005, p. 5). One of the major 
reasons behind this disparity was that although “a low-income household spends, on average, 9 
percent of its income on secondary education, a high-income household spends only 1.7 percent” 
(Calderon, 1996, p.2). This has had far-reaching implications for low-income household members 
who were often unable to find places for their children in already overcrowded public schools and 
unable to afford private schools. Furthermore, according to Calderon (1996), the ability of high- 
and higher middle-income families to send their children to private schools “had the serious effect 
that the dominant social and economic sectors [had] shown no personal interest in the quality and 
development of public education” (p. 2). Ultimately, it appears that the call of global competition 
caused the government of Colombia to refocus its effort on rebuilding the education system in 
the 1990s, paying particular attention to increasing secondary school enrollment. 

In March 1991, with a public school system at capacity and a public infrastructure unable 
to meet the demand for public schools, it became apparent that if the government was serious 
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about making significant changes, a new approach was necessary. Consequently, in the midst 
of a national transformation toward decentralization, which brought about “shifting power from 
the central government to local institutions and [an] expanding role of the private sector in the 
provision of public services” (p. 9), Education Minister Alfonso Valdivieso began working on a 
voucher program (King et al., 1997). Within a year, Valdivieso’s proposed plan was approved and 
the Programa de Ampliaci6n de Cobertura de la Educacién Secundaria (PACES) was initiated in 
late 1991, with support from the World Bank (Angrist et al., 2002; Calderon, 1996; King et al., 
1997). 

Officially introduced as part of the 1992 Decentralization Bill, PACES awarded more than 
125,000 vouchers before being discontinued in 1998 (Angrist et al., 2002). Specifically targeting 
students in the lowest two economic quintiles, PACES sought to increase secondary enrollment 
“by exploiting the existing strong private infrastructure to expand capacity quickly” (Mayer, 
n.d., p. 7). To avoid criticism that the voucher plan would threaten public school resources, “the 
government assured public schools that current levels of funding would not decrease” (King, 
Orazem, & Wohlgemuth, 1999, p. 471). Also, by targeting only low-income students, PACES 
effectively countered “the claim that vouchers amount to a net subsidy for the wealthy at the 
expense of the poor” (King et al., 1999, p. 471). Furthermore, 


in an attempt to counter fears of politicization and central government management which would 
discourage participation, [PACES] was administered through the Instituto Colombiano Educativo y 
Estudios Tecnicos en el Exterior (ICETEX), which had previously managed study abroad programs, 
and voucher payments were made directly through the Banco Central Hipotecario (BCH). (Mayer, 
n.d., p. 7) 


Ultimately, the main objectives of PACES, as posited by Calderon (1996), were to 


(i) close the gap between the relatively high primary school enrollment rate and the relatively low 
secondary enrollment rate ... (ii) to expand the very limited choices that the poorest families had in 
relation to secondary education for their children, and (iii) to improve equity. (p. 3) 


It is with these objectives in mind that we now turn to an in-depth look at the design and 
structure of the PACES voucher plan. Following the comprehensive structure for examining the 
design instruments central to any voucher plan—as defined by Levin and Belfield (2005)—the 
following section looks at the finance, regulation and support services of PACES. 


FINANCE 


By averaging the tuition fees of low-to-middle cost private schools in Colombia’s three largest 
cities (i.e., Bogota, Medellin, and Cali), PACES established a fixed-price for its vouchers (Angrist 
et al., 2002). There are, however, two important points that must be acknowledged here: (a) if 
the worth of the voucher exceeded that of the tuition of a given private school, the school would 
only receive the amount of their regular tuition, and (b) if families wished to send their children 
to private schools with fees exceeding that of the set voucher amount, they were permitted to 
supplement the voucher with personal contributions (Angrist et al., 2002; Kremer, 2002). 
While relying on World Bank loans to subsidize approximately 5,500 vouchers per year, 


the Ministry decided that the central government would pay for 80 percent of the cost of each 
voucher and the cities would cover the other 20 percent. [Furthermore,] most mayors were interested 
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in expanding secondary schooling for a fraction of its cost and hence became involved and helped 
promote it. (Calderon, 1996, p. 5) 


With regard to the actual payment of the voucher, PACES relied heavily on the aforementioned 
BCH. While the Colombian Social Investment Fund was responsible for channeling the govern- 
ment’s share of payments, participating schools received their funds directly through BCH (King 
et al., 1997). As a matter of fact, 


school principals were instructed to open an account, free of charge, at any of the 120 BCH branches... . 
[and] every three months, upon presentation by the school of the grades of the voucher beneficiaries, 
the school’s account would be-credited in the appropriate amount. (Calderon, 1996, p. 8) 


Generally, the first two payments of the year (along with a small portion of the third payment) 
consisted of the central government’s contribution, whereas the remainder of the third payment 
was from the respective municipalities. Furthermore, “when students drop[ped] out in the middle 
of the academic year, an adjustment [was] made in the remaining annual payments to the school” 
(King et al., 1997, p. 14). (Note: Additional financing and administrative cost information is 
examined further in the section on productive efficiency.) 


REGULATION 


Regulatory measures were established for PACES with regard to student, school, and municipality 
participation. Because students’ eligibility was relevant only if they had access to an eligible 
school—and schools’ eligibility relied on their location within a participating municipality—it 
seems most appropriate to analyze the regulation requirements beginning at the municipal level. 

“Municipalities of 10,000 or more inhabitants, located less than 50 kilometers from a depart- 
mental capital and with three or more private schools were eligible to participate” (Mayer, n.d., 
p. 9). In addition, “each municipality decided how many vouchers to fund, subject to a maximum 
allocated to towns by the central government. This allocation was determined by estimating the 
shortfall between primary-school enrollment and the available space in public secondary schools” 
(Angrist et al., 2002, p. 1538). When the demand of voucher applicants exceeded the maximum 
municipal allocation, a lottery system was implemented—and in certain cities with excess sup- 
ply, such as Bogota, ICETEX even “provided software and instructions to regional offices for the 
purposes of random selection of applicants” (Angrist et al., 2002, p. 1539). Ultimately, “munici- 
palities choosing to participate were required to maintain a system of accounts set according to 
a nationally prescribed standard ... [whereas] ICETEX was to be responsible for regulation of 
municipalities and their participation” (Mayer, n.d., p. 9). 

At the school level, the most prominent regulation was that only those licensed schools in 
participating municipalities that offered a traditional academic or vocational curriculum were 
eligible to receive vouchers (King et al., 1997). Also of great significance was the fact that 
PACES vouchers were limited to private schools. However, although participating private schools 
initially “included for-profit schools, religious-affiliated schools, and schools run by charitable 
foundations” (p. 1538), by 1996 for-profit schools were excluded from PACES (Angrist et al., 
2002). To mitigate against schools increasing their fees as a result of the voucher—once initial 
eligibility was determined on the previously stated criteria—all schools were required to “provide 
information on their matriculation and tuition fees at the time of entry” (King et al., 1997, p. 12). 
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Also, according to King et al. (1997), in addition to regular monitoring of participating schools 
by ICETEX, “to regulate the quality of participating schools, new schools that [were] interested 
in the program [were] required to present a three-year curriculum plan” (p. 12). Resultant from 
these regulations—as is discussed further in the sections on scope and equity—”participating 
schools tended to serve lower-income pupils, and to have lower tuition than nonparticipating 
private schools ... [and] schools with a vocational curriculum were also overrepresented among 
those in the program” (Angrist et al., 2002, p. 1538). In addition, Restrepo (2004) noted that 
poor monitoring of these schools had a significantly negative effect on both the short-term and 
long-term objectives of the program. 

As previously stated in this article, PACES was a targeted voucher program for students 
entering secondary school, namely, sixth graders younger than 15 (Kremer, 2002)—in the lowest 
two economic strata of six, based on a neighborhood stratification scheme (King et al., 1997). 
Therefore, “applicants had to submit a utility bill to establish residential location and voucher 
eligibility” (Angrist et al., 2002, p. 1537). In an attempt to avoid subsidizing students who 
would have attended a private secondary school regardless of voucher status (King et al., 1997), 
according to Calderon (1996), the government required families “to show proof that their children 
had finished primary school at a public school” (p. 6). A final regulation regarding student 
participation was that vouchers were to be automatically renewed each academic year until 
graduation from secondary school, provided a student did not fail/repeat any grade. In other 
words, if a voucher recipient was forced to repeat a grade, he or she was to be dropped from the 
program automatically (Angrist et al., 2002; Calderon, 1996; King et al., 1997). 


SUPPORT SERVICES 


Although a significant component of any successful voucher program, there is very little infor- 
mation regarding support services in the PACES literature. Most apparent is the complete lack of 
information regarding transportation. Both in terms of cost and access, transportation should be 
considered an integral aspect of the voucher system design process—especially in the implemen- 
tation of a system with selection criteria such as income, based on neighborhood stratification. 
For, as Mayer (n.d.) asserted—citing King et al.’s (1999) usage of tuition as a proxy for academic 
quality—“if better schools are more expensive schools, they are less likely to be in poorer areas. 
If students wish to upgrade to these schools they will most probably face increased transportation 
costs” (p. 10). 

Calderon (1996) did, however, note that support services were not lacking altogether. For 
example, “the main characteristics of the program were made public during visits by the Minister 
and the Vice-Minister of Education to each of the program cities, and were advertised in the local 
newspapers and radio stations” (p. 6). For parents interested in enrolling their children in PACES, 
voucher applications were made available both through newspaper advertisements and at all 
regional ICETEX offices. Furthermore, families/neighbors/local organizations could share in the 
responsibility of acquiring and disseminating these applications, as they could be photocopied and 
distributed accordingly. However, despite modest attempts to facilitate the application process, 
the support services for PACES ultimately left much to be desired. 

With the design instruments of finance, regulation, and support services clearly defined for 
PACES, this article now focuses on an in-depth evaluation of the Colombian voucher program, 
analyzing the extent to which it was successful in increasing choice and enrollment opportunities 
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for low-income students, its productive efficiency, and the ultimate effect it had on closing the 
achievement gap. 


ACCESS AND CHOICE 


Complete freedom of choice could be defined as a situation where students could attend any school of 
their choosing, unfettered by economic, transportation or other constraints, and where schools would 
be considerably distinct in their character so as to make that choice significant. (Mayer, n.d., p. 12) 


Levin (1991) further asserted that subcomponents of choice and access include things such as 
magnitude and range of school costs covered by vouchers, the range of choices with regard to 
school philosophies, the degree of regulation of curriculum and admissions, and the short-run 
and long-run supply of schools. With these components at the crux of the debate, did PACES 
truly provide considerably increased access and freedom of choice for a significant number of 
secondary school students in Colombia, thus decreasing the gap between rich and poor students? 

Although PACES provided more than 125,000 students with vouchers over 7 years, it is 
important to note that this number accounts for approximately one tenth of the private school 
secondary enrolment in Colombia and only about 4% of the total secondary school enrolment 
(Bettinger, 2005, p. 5). In addition, according to King et al. (1997) only 216 (20%) of the 
country’s municipalities participated in PACES, further limiting the range of students affected 
by the program. More specifically, “large urban municipalities with greater numbers of primary 
school students, more limited public school capacity, and larger existing private school capacity 
[were] the most likely to participate in the program” (p. 4). Approximately 4 to 5 years into the 
program, issues surrounding decreased quality and political opposition complicated matters even 
further. 

In 1995, in Bogota, for example, 15% of the 12,000 voucher students were in “new” 
schools—most of which had problems with quality and many of which subsequently closed. 
This had serious implications with regard to choice because, as Calderon (1996) noted, “parents 
who send their children to the new schools usually have no other option—they do not have access 
to public schools nor the means to pay for any extra tuition in private schools” (p. 13). In addition 
to many new schools being shut down and concomitant with the exclusion of for-profit schools 
from PACES in 1996, choices became narrowed further due to the ministry of education’s unwill- 
ingness to provide any vouchers beyond the 5,500 subsidized by the World Bank. Consequently, 
with a constant demand and a diminishing supply, ICETEX reported that municipalities such as 
Atlantico were only able to provide approximately one fifth of their qualifying applicants with 
vouchers (King et al., 1997). 

With regard to choice and the rich—poor enrollment gap in developing countries, King et al. 
(1999) asserted that “education choice is not whether to attend public school or private school, but 
whether or not to attend school at all” (p. 468). Accordingly, it has been argued that the provision 
of vouchers opened up spaces for out-of-school youth in otherwise full public schools. Kremer 
(2002), however, noted that many voucher recipients in Colombia were planning to enroll in 
private schools regardless; therefore, it appears that the program’s ability to reduce the enrollment 
gap was diminished, with only a small number of low-income students now able to attend more 
expensive (though arguably higher quality) private schools. Ultimately, although PACES may have 
increased choice for a small percentage of the secondary school population—with no mention of 
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special needs students, no provisions for transportation (and little in the way of support services 
on the whole), the exclusion of for-profit schools and a limited voucher supply—it does not 
appear that access and choice were, on the whole, significantly increased. However, to more 
fully understand whether the secondary enrollment gap was truly decreased, we now turn to an 
examination of the issue of equity in access. 


EQUITY 


According to Belfield and Levin (2005), equity refers most broadly to “the quest for fairness 
in access to educational opportunities, resources, and outcomes by gender, social class, race, 
language origins, disability, and geographical location of students” (p. 35). More specifically, 
Levin (2002) noted that the “components of analysis include the degree of access to educational 
opportunities, the quality of those opportunities including school resources and peers, and the 
probable educational outcomes for those groups” (p. 27). With regard to PACES, it is neces- 
sary to look at (a) the actual population served by the program (did students from the lowest 
economic strata actually receive more equitable access to educational opportunities?), (b) where 
these students came from geographically, and (c) the quality of participating schools (were the 
teacher/school characteristics equitable when compared with public schools and nonparticipating 
private schools’). 

There are several issues to consider when attempting to answer these questions. First, PACES 
was officially launched in Colombia’s 10 largest cities (Calderon, 1996). By limiting access to 
vouchers to students residing in the largest cities in Colombia—and to participating munici- 
palities which had higher tax revenues (Angrist et al., 2002)—a considerable percentage of the 
poorest families, residing in rural areas of Colombia, were effectively excluded from participa- 
tion. Second, applicants were required to present utility bills as proof of residency as a proxy 
for income (King et al., 1997). Although this requirement would limit applicants to those living 
in certain targeted neighborhoods, the assumption was that all families would have the ability 
to provide a utility bill—which would not always be the case for the poorest households. Third, 
“those interested had to fill out the vouchers included in newspaper advertisements or could 
obtain free application forms at any of the regional ICETEX offices” (Calderon, 1996, p. 6). 
Despite providing multiple means through which parents could obtain applications for PACES, 
the idea of application forms in newspapers (or even at ICETEX offices) assumed a certain level 
of literacy that may not be found among adults in Colombia’s poorest households. Finally, all ap- 
plicants were required to have already been admitted into a participating private school and many 
participating students reported that they would have attended private school whether or not they 
received the voucher (Angrist et al., 2002). In addition to Arenas’s (2004) concern that schools 
are “entitled to reject students through a selective admissions process .. . [and] it is quite possible 
that children considered undesirable have less chance of being admitted than easy-to-educate 
students” (p. 390), there are also serious implications for this requirement with regard to the type 
of student who even applied to these private schools. Because vouchers covered only about half 
of the private school fees for participants (Angrist et al., 2002) and, as Kremer (2002) noted, 
“no one is on the private school/no school margin, and so small subsidies to private education 
do not directly increase overall enrollment” (p. 3), it becomes quite apparent that students from 
the poorest households would not have applied to private schools in the first place and would 
therefore not have been served by PACES vouchers. 
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What is known about the quality of the schools that PACES students chose to attend? King 
et al. (1997) asserted that “there are no statistically significant differences between the average 
mastery levels of students in participating private schools and students in public schools in the 
field of language and mathematics” (p. 33). Angrist, Bettinger, and Kremer (2006) further note 
that “pupil—teacher ratios and facilities were similar in public and participating private schools, 
and many of the teachers in the private schools that were most likely to participate in the PACES 
program were moonlighting or retired public school teachers” (p. 4). Although these findings 
indicate that voucher students were presented with equitable options in participating private 
schools as compared with the public schools they were leaving behind, a deeper examination 
shows that this was not necessarily the case. For example, Mayer (n.d.) noted that public school 
teachers were actually paid twice as much as PACES teachers, averaged twice the amount of 
time in school and had more training. In addition, Calderon (1996) offered that quality problems 
surfaced in many of the “so-called new private schools” and that “during the first stages of 
the program, lack of control led to the creation of some very deficient schools” (p. 13). In a 
remark directly contradictory to the argument that PACES increased equity for voucher recipients, 
Calderon added, “Generally, the parents who enrolled their children in such schools had no other 
option” (p. 13). With regard to equity between participating and nonparticipating schools, Kremer 
(2002) noted that “participating schools tended to serve lower-income pupils, and to have lower 
tuition than non-participating private schools. Relatively elite private schools appeared to opt-out 
of the PACES program” (p. 2). Furthermore, “there were other indications that the quality of 
program schools was lower on average than non-program schools. Participating schools were 
more likely to offer nonacademic (vocational or technical) tracks and had higher pupil—teacher 
ratios than did nonparticipating schools” (King et al., 1999, p. 485). Finally, Mayer (n.d.), citing 
Ribero, Tenjo, and Sanabria (2001), offered that “PACES schools were scored high quality only 
11% of the time, as compared with ... 51% for non-participating schools on government ratings” 
(p. 18). In light of these findings, it would be difficult to argue that equity for low-socioeconomic 
status, secondary school students was truly advanced by the introduction of PACES or that there 
was any significant decrease in the enrollment gap between the wealthiest and poorest students 
in Colombia during this time. 

The following section explores the academic aspect of the achievement gap, as well as the 
cost-effectiveness of PACES in its attempt to narrow both the academic and enrollment gap for 
secondary school students in Colombia. 


PRODUCTIVE EFFICIENCY 


In its simplest form, productive efficiency refers to the maximization of (educational) outputs for 
any given input. Although a seemingly straightforward and self-explanatory definition, the idea 
of productive efficiency is slightly more complex and nuanced than it first appears. First, the term 
educational outputs must be deconstructed. Educational outputs is too often used interchange- 
ably with standardized test scores—but as Levin (2002) asserted, “academic achievement should 
include not only knowledge as measured on test scores, but also understanding and application of 
knowledge in real-world situations” (p. 27). In addition, educational “outcomes should include 
not only the standard academic subjects, but also other skills such as problem-solving, working 
in teams, and effective decision-making” (Levin, 2002, p. 27). Gill, Timpane, Ross, and Brewer 
(2001) furthered that “‘studies should include measures that reflect the richer set of academic 
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outcomes that schools are expected to produce ... including continuation in school, graduation, 
and college attendance” (p. 113). Accordingly, the evaluation of PACES with respect to pro- 
ductive efficiency will incorporate a variety of educational outcomes—beyond standardized test 
scores—along with a few noneducational outcomes, such as hours worked and future earnings 
potential. 

With regard to the cost function, Levin and Driver (1997) proposed a wide array of components 
necessary for consideration, including accommodation of additional students, record-keeping 
and monitoring systems, transportation costs, information costs, and costs of adjudication. “Also, 
there must be a distinction between the costs of producing education at the school site and those 
associated with the system of monitoring and administering the educational system and providing 
support services” (Levin, 2002, p. 27). The following analysis of cost-benefits of PACES will 
include these criteria as well as direct and opportunity costs to families. 

Due to the excess demand for PACES subsidies and the subsequent lottery assignment of 
vouchers, several natural experiments have been carried out to compare achievement results on 
standardized tests for lottery winners and losers.' For example, Angrist et al. (2002) administered 
La Prueba de Realizacion (a multiple-choice achievement test for native Spanish speakers) to 
just fewer than 300 lottery winners and losers in Bogota and found that voucher recipients scored 
just over 0.2 standard deviations higher than lottery losers. To put this number into perspective, 
“according to U.S. norms for La Prueba, two-tenths of a standard deviation is roughly the score 
gain associated with one additional school year” (Angrist et al., 2002, p. 1545)—which Angrist 
noted is significant considering these tests were administered to students who had been in voucher 
schools for only 3 years at the point of testing. Although these gains are impressive, it is important 
to note that not only is the sample of students quite small but also it is difficult to claim that a 
sample from Bogota would be representative of the secondary school population at large. 

The results of a second standardized test have also been used to analyze the effectiveness of the 
voucher system. Angrist et al. (2006) looked at results of Colombia’s centralized college entrance 
exam—also known as the ICFES exam—and found that prior to adjusting for censoring, lottery 
winners scored less than | point higher on both the language and math exam, with similar effects 
for boys and girls. Mayer (n.d.), citing Ribero et al., (2001), noted that these gains are statistically 
insignificant. Although Angrist et al.’s (2006) findings with regard to ICFES test scores show 
neither an advantage nor a disadvantage to voucher schools, their analysis of registration results 
for the exam point to modest increases in attainment for voucher recipients. 

Using registration for the ICFES exam as a proxy for secondary school graduation rates, 
Angrist et al. (2006) found that after controlling for demographics, lottery winners are about 
6 percentage points more likely to take the college entrance exam. In a more recent study on 
longitudinal effects, Saavedra, Bettinger, Kremer, and Kugler (201 1) found that voucher recipients 
were 10% more likely to access higher education. In addition, Angrist et al. (2002) noted that 
in their full sample, lottery winners completed 0.8 more years of schooling than lottery losers. 
Although Kremer (2002) furthered that “lottery winners were about 10 percentage points more 
likely than losers to have completed 8th grade” (p. 4)—Mayer (n.d.), citing Patrinos (2002), 
asserted that this advantage may be as high as 13% to 15%. Bettinger (2005) argued that a 
25% reduction in repetition rates for lottery winners contributed to these increased levels of 


"It should be noted that this is not a direct public versus private school debate, as many lottery losers still chose to 
attend private school (at least for | year). 
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attainment. However, Kremer noted that “separate results by sex show moderately larger effects 
on educational attainment for girls ... [and these effects] appear to come from both reduced 
grade repetition and additional time spent in school” (p. 4). Angrist et al. (2002) asserted that the 
additional time spent in school by girls is directly related to the fact that marriage and cohabitation 
rates were lower for lottery winners and the finding that lottery winners worked approximately 
1.2 fewer hours per week. Although attainment apparently increased—at least in part—as a result 
of voucher students working fewer hours during the school year, an examination of the rate of 
return for schooling will be undertaken in the following section on costs. 

As previously mentioned, the costs of a voucher system are multidimensional—this analysis of 
the costs of PACES will begin with voucher tuition coverage and direct familial costs. Although 
PACES vouchers were initially designed to cover an annual matriculation fee and ten monthly 
tuition payments (King et al., 1997), by insufficiently adjusting for inflation, vouchers only 
covered about half of the voucher recipients’ school fees by the mid-1990s (Angrist et al., 
2002). More specifically, according to Angrist et al. (2002), lottery “winners’ gross school fees 
exceeded those of losers by about 70 percent of the amount they received from the voucher” 
(p. 1536). Furthermore, this $52 increase in expenditures combined with a calculated $41 of lost 
earnings, led Angrist et al. (2002) to “estimate that PACES lottery winners devoted $93 more to 
education than losers ... or 126 percent of the $74 in extra scholarship assistance they received” 
(p. 1551). Kremer (2002) asserted, however, that as voucher recipients completed an additional 
.12 to .16 grades—based on a recent Colombian labor force survey—winners’ average wages 
should increase by 1.2% per year (or even by as much as 10% if the calculation is based on the 
achievement gain from standardized tests). With expected yearly earnings of applicants hovering 
around $3,000, Kremer posited that “PACES seems very likely to raise lottery winners’ wages 
by $36, and might raise wages by as much as $300 per year if higher test scores have a grade- 
equivalent payoff” (p. 7). Saavedra et al. (2011) similarly find that PACES vouchers increased 
monthly incomes for beneficiaries by an average of 10%, giving the program a cost—benefit ratio 
of 7:1. Although Kremer also argued that the benefit of increased future earnings should “easily 
outweigh” the additional costs of schooling, it is important to note that in a situation where parents 
earn an average of only $2,400 per year, increased expenditures in the short term might not be 
feasible, regardless of the potential long-term benefits. 

Although it is important to look at the costs of PACES for participants, in order to be able 
to draw any legitimate conclusions about its productive efficiency it is absolutely essential to 
examine the actual costs of administering the program. With relation to the public school system, 
“the voucher program cost the government about $24 more per winner than the cost of creating 
a public school placement” (Angrist et al., 2002, p. 1536). Furthermore, King et al. (1997) noted 
that 


ICETEX charges a commission fee of 3.5 percent of the central government’s share of the cost of the 
program. The Social Investment Fund charges less than 0.2 percent of the total cost of the program 
and, for each of the three transactions carried out annually with a participating school, the Central 
Mortgage Bank charges 1,000 colombianos. (p. 16) 


In addition, although King et al. (1999) asserted that “the monitoring system, even for a geo- 
graphically targeted program, proved to be quite costly” (p. 489), Calderon (1996) argued that 
schools relying on vouchers should have been subject to even closer monitoring—which would 
have increased the costs even more. Further funds would also have been necessary to more closely 
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monitor the payment system to prevent payment delays and to continue providing vouchers in ad- 
dition to those 5,500 funded by the World Bank. Although the literature on PACES provides some 
insight into the pertinent program costs (all of which point to increased costs for both participants 
and the government), there is no mention of costs for support services such as dissemination of in- 
formation, transportation, and training. Balancing the mixed (or even slightly positive) results on 
student achievement, attainment, and potential increased earnings with the increased costs across 
the board (even without accounting for support services), it could be argued that PACES was 
able to slightly decrease the achievement gap between low-income voucher recipients and their 
middle-income classmates, but it does not appear that this was accomplished in a cost-effective 
manner. 


NONACADEMIC OUTCOMES 


Undoubtedly the most difficult aspect of any educational programming to quantify, nonacademic 
outcomes are nonetheless essential factors in analyzing a voucher program. Although there is 
little or no mention of nonacademic outcomes in the majority of the PACES literature, this article 
provides at least a modest insight into several factors that may contribute to the overall effect of 
PACES on these criteria. 

One major concern of voucher opponents is that the implementation of a voucher program will 
eventually lead to greater segregation by socioeconomic status. Arenas (2004) noted, however, 
that because “only low-SES students are voucher-eligible . . . [and] Colombian public schools have 
historically enjoyed relatively high prestige, particularly when compared with low-tuition private 
schools ... thus far, Colombia has been able to minimize segregation” (p. 389). As a matter 
of fact, Angrist et al. (2002) argued that voucher recipients actually attended more expensive 
private schools than would have been possible without the voucher, subsequently increasing the 
socioeconomic status range in participating schools. It is important to note, however, that this 
range was not necessarily as significant as we may be led to believe, as the most prestigious (i.e., 
expensive) private schools chose not to participate in PACES. 

With regard to what Arenas (2004) called civic socialization, it should be noted that com- 
munity service is mandatory for both private and public school students in Colombia. Although 
further research would be necessary to determine whether private or public school students spent 
more time participating in community service activities, PACES literature shows that in-school 
participation seems to have increased slightly for voucher students. For example, Mayer (n.d.) 
asserted that “students in the program were more involved in their education, had more knowledge 
of curriculum, and felt as if they were listened to when they voiced concerns” (p. 20). 

Unfortunately, there is scant evidence in the literature on other, important nonacademic out- 
comes such as school safety, job placement and any significant measures of social cohesion (both 
in and out of school). Ultimately, there is insufficient evidence to make an informed judgment 
regarding the effect that PACES had on many of these essential, nonacademic outcomes. 


CONCLUSION 


Introduced in late 1991, PACES provided vouchers to more than 125,000 low-income secondary 
school students throughout its approximate 7 years in operation. After many political struggles 
and decreased funding in the final years, “in 1998, Colombia dissolved the voucher program. 
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A new administration did not see the program as central to its educational initiatives” (Bettinger, 
2005, p. 7). “The then director of the National Planning Department criticized the reform for 
its unfulfilled promises and politicization, reiterating the need for an expansion in the education 
sector if the country [was] to sustain its growth” (Mayer, n.d., p. 20). 

Although PACES was successful in providing a relatively small number of low-income sec- 
ondary school students in Colombia with the opportunity to attend private schools that might 
otherwise have been too costly, by limiting school options to less than half of the nonprofit pri- 
vate schools in major cities, not addressing the issue of transportation and only providing vouchers 
for 5,500 students in the program’s final few years, PACES never produced its desired results. 
With regard to enrollment objectives, although PACES targeted the two lowest economic strata for 
participation, due to an overrepresentation from the second strata, along with inequitable access 
to schools of the highest standards, PACES did not produce its intended effect of significantly 
decreasing the secondary enrollment gap by increasing choice for low-income students. 

Therefore, if one of the goals of a voucher program was to increase overall enrollment rates 
by either freeing up public school spaces or providing vouchers to those most in need, modeling 
a program on PACES would not be advisable. Attaining this goal would require more accessible 
information for parents, including assistance with voucher applications (particularly in areas with 
low literacy rates), greater investments in transportation for students, increases in the number 
and types of schools accepting vouchers, considerations of the proportion of school fees to be 
covered by the voucher, increased monitoring of schools accepting vouchers (with a particular 
focus on new schools established in order to take advantage of the voucher system), and stricter 
regulations regarding voucher recipients (assuming a fixed number of spaces, as opposed to a 
universal voucher program). 

As for academic achievement, voucher lottery winners in Colombia showed several signifi- 
cant gains over their naturally randomized comparison group. Voucher recipients saw a higher 
likelihood of finishing eighth grade, an increase in years of schooling and high school graduation 
rates, as well as a reduction in repetition rates and slight increases in achievement test scores. 
Although the ultimate scope of the program disallows us from making more large-scale claims, 
these findings support the claim that the PACES program was at least partly effective in decreasing 
the achievement gap between low- and middle-income students in Colombia. 

With regard to productive efficiency, although Angrist et al. (2002) and others have claimed 
that vouchers increased the quality of education received by their recipients, it is important to 
note that 


quality has been measured by standardized test scores and rates of retention, promotion and graduation. 
Other key aspects of quality (e.g. how well a school fosters in students emotional well-being, physical 
and artistic development, and a sense of empathy for others) have not been studied because of the 
difficulty of measuring and standardizing the results. Given that quality is much broader than what is 
currently measured, the results provided for this dimension require cautious interpretation. (Arenas, 
2004, p. 382) 


Furthermore, even if such stated academic achievements were to be taken at face value, with 
the increased costs to both families and society, it would be difficult to argue that PACES provided 
greater productive efficiency than the existing system—and this is without the additional services 
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previously recommended. Finally, although minor claims can be made about in-school partici- 
pation, there is insufficient evidence to determine PACES’s true effect on critical, nonacademic 
outcomes. 

Although PACES no longer exists in Colombia, this analysis provides important lessons for 
future policies. Despite the fact that PACES had several significant shortcomings, its successes 
strengthen the argument that implementing an income-contingent voucher program can help to 
reduce the academic achievement gap in developing countries. However, future reforms would 
undoubtedly require significant modifications and expansions to their financing and regulations 
if they are expected to produce meaningful gains in enrollment—and if they wish to do so in a 
widespread and cost-effective manner. 
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The international Education for All initiative to bring about universal primary education has resulted 
in large enrollment increases in lower income countries but with limited outcomes. Due to scarcity 
in material and human resources, all but the better off often fail to learn basic skills. To improve 
performance within the very limited capacities of low-income educational systems, instructional 
interventions ought to be designed according to the ways people retain and recall information most 
efficiently. Without this research-based line of reasoning, donor and government staff may overesti- 
mate students’ ability to learn from complex methods and scant practice. Applicable concepts include 
perceptual learning for scripts, automaticity for basic skills, the limitations of working memory, and 
formation of cognitive networks in long-term memory. However, education faculties rarely teach 
these topics. A systematic effort is necessary to popularize cognitive science concepts pertinent to 
basic skills for staff working in the education sector of lower income countries. Therefore, better 
understanding and application of learning research is urgently needed if universal primary education 
is to succeed. 


The 1990 Education for All initiative has resulted in an explosion of activity that has continued 
unabated in the last two decades (UNESCO, 2000). It has resulted in establishing a United 
Nations Millennium Development Goal of universal primary education. The “for all” expression 
indicates the goal for countries to educate every student, regardless of achievement level or status, 
aside perhaps from certain special cases. The emphasis to educate all children has challenged the 
educational systems of poorer countries to accommodate many additional students with severely 
limited budgets. In these new circumstances, decisions may challenge the staff of governments 
and consultants who advise them. What choices should be made and why? Some examples are 
next. 


e Countries with limited infrastructure and teacher availability, like Senegal, Niger, or 
Mozambique, were advised to reduce class sizes by halving instructional time and in- 
stituting multiple shifts. Common sense would suggest that most students would not master 
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the intended curricula, particularly given the need to learn French, English, or Portuguese 
first. Some educators and donor staff protested, but they could not offer convincing argu- 
ments or alternatives. Eventually it was presumed that students would somehow “absorb” 
the curricular content (World Bank, 1999, p. 29). Subsequently an article in a Nigérien 
newspaper stated that to succeed in school a student had to be either gifted or well off 
(Boubakar, 2003). 

e To prevent massive dropout and repetition due to instruction methods, policymakers in 
low-income countries were advised to institute automatic promotion, sometimes up to the 
end of primary school. Again, some educators sensed that children would not learn in higher 
grades what they did not learn in the lower ones. Social promotion has become a worldwide 
policy, despite the lack of a logical understanding of how people learn. This resulted in 


widespread illiteracy among graduates of some countries, such as Togo and Niger (Mingat, 
2003). 


A Minister of Education in Central America decided to change reading textbooks from phonics 
to the whole language approach. The minister thought that it was worth waiting for the new 
textbooks because students ought to learn reading through generative words and authentic texts. 
Similarly, a donor agency advised an African country contemplating curricular changes to save 
money by not reprinting existing textbooks until new curricula and new textbooks were available. 
Without textbooks, much of classroom time is spent copying from the blackboard. Many students 
sketch leiters, the meaning of which they do not understand, or may copy text too slowly to make 
sense of it. 

How could better decisions be made for problems such as the preceding? How could classroom 
time be used to enable basic skills acquisition? Staff who work in government offices and donor 
agencies often have no clear answers. Many have varying academic backgrounds, from literature 
to economics, and may have only their own schooling experience as a frame of reference. They 
may thus recommend general ideas such as “redefining goals and targets, curricular design and 
content, educational methods and classrooms, partnerships to address equity and inclusion.” Some 
documents even refer to classroom instruction as a “black box.” International organizations that 
offer plentiful advice on financing and organizational matters have been reticent about instruction. 
In absence of relevant knowledge, donor strategies focus on peripheral factors. For example, 
the World Bank’s (2011) “Learning for All” strategy focuses on making school authorities 
autonomous and accountable in expectations that they will find means for teaching students 
efficiently. 

As a result of such processes, instructional strategies are often viewed as details and are left 
up to the discretion of teachers and local authorities. It is they who ultimately determine the 
effectiveness of billions of dollars in aid. However, many teachers are not qualified enough to 
create and execute effective instructional strategies. In the lowest income countries, they may have 
the equivalent of a fourth-grade education. Efforts to train them based on techniques imported 
from high-resource countries have been unsuccessful (Craig, Kraft, & du Plessis, 1998). 

Due to a multiplicity of languages, instruction in many countries takes place in English, French, 
or Portuguese. These languages, along with their complex spelling systems, must be learned 
during class. Teacher absenteeism is often around 20%; overall, less than half of the allocated 
instructional time may be used for instruction (e.g., Abadzi, 2006, Schuh Moore, DeStefano, 
& Adelman, 2009). Parents are often illiterate and cannot help their children. Textbooks are 
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usually imported, so they tend to be expensive and scarce. Thus, students may spend the majority 
of class time copying incomprehensible texts from the blackboard. In addition, students face a 
host of obstacles known to compromise learning, such as malnutrition and infectious diseases 
(Grantham-McGregor et al., 2007). 

It is not surprising, therefore, that students who entered school at great expense often fail to 
attain even minimal competencies (Filmer, Hasan, & Pritchett, 2006; UNESCO, 2005). Reading 
fluency tests in the primary grades show that in some countries 90% of the students cannot read 
even individual letters (Research Triangle Institute [RTI], 2010a). Many drop out or even graduate 
illiterate. 

Several government and donor staff earned education degrees in higher income countries yet 
wouldn’t necessarily be equipped to provide applicable insights. Those interested in international 
education often get master’s degrees or doctorate degrees in fields such as policy, finance, 
educational sociology, adult education, or comparative education. These programs often focus 
on topics peripheral to learning; students may not be required to take any courses in instructional 
methodology and learning principles or to obtain teaching experience.' Graduates who are hired 
for positions of “education specialists” in fact may have little specialized knowledge. They may 
recommend a child-friendly classroom climate, innovative methods, constructivist classrooms 
where students do not memorize, individualized instruction despite class sizes. Citizens of low- 
income countries who receive advanced degrees from education faculties of Europe or the United 
States may give poor advice to their governments. They may espouse policies that are unrealistic 
for low-income students, such as whole-language Grade | textbooks. 

In some respects, the idealist suggestions of donor staff are not surprising. In higher income 
countries, the educational services are usually of sufficient quantity and quality to help even low 
performers acquire basic skills. Children helped by middle-class parents can carry out complex 
tasks such as reading acquisition by whole-language approach. In poorer countries, however, the 
initial deficits are large. Many children start school without having seen a book or held a pencil, 
and they may have limited vocabularies even in their own languages (e.g., Hart & Risley, 1995; 
Paxson & Schady, 2005). 

Imagine, therefore, a Grade | in a semirural area of a low-income country. Due to absenteeism 
and need to fit multiple shifts, the students will get only 20% of the instructional time given to 
first graders in higher income countries. The students have slates for writing but no textbooks. 
The teacher has learned from long experience to interact mainly with the few best students who 
can retain large chunks of information with limited exposure (Abadzi & Llambiri, 2011). What 
changes are feasible to get as many literate students as possible? Government officials could 
potentially act on a few clear pieces of advice, but which ones would give the greatest benefit? 

One way of setting priorities practically is to consider human cognition. Memory evolved to 
help organisms survive over the millennia, so humans from Laos to Argentina are genetically 
set to retain and organize information through the same general processes (Huitt, 2003). The 
cognitive commonalities are most evident at the more basic stages of information processing: 


'For example, the background information for the master’s degree in international and comparative education of 
Stanford University stated in 2011: “Students in the program examine educational policy issues in an international 
context. Working closely with professors from a variety of disciplines, they study such problems as educational planning 
in comparative perspective, the dynamic relationship between school and community, equity and education, and the 
political economy of underdevelopment” (http://ed.stanford.edu/academics/masters). 


CHALLENGES AND HOPES OF COGNITIVE NEUROSCIENCE 61 


visual perception and discrimination, encoding, recall, retrieval, and use of information. A chain 
of causality based on this body of research would predict likely outcomes for various classroom 
events and provide insights about making instructional methods more efficient. 

This causal chain is derived from cognitive science (or cognitive neuroscience).” Teaching 
humans in the ways they were set up to learn may help deliver education to the poor more 
efficiently and thus minimize the limitations of low-income school systems. Curricula, textbooks, 
teacher training, and educational policies could be designed on the basis of this causal chain. 

However, this domain of knowledge is little known outside psychology departments. Edu- 
cation faculties rarely teach it, and few educational psychology textbooks emphasize memory 
and perceptual functions (e.g., Davis, Sumara, & Luce-Kapler, 2007). There have been some 
exemplary efforts in applying the relevant knowledge to schools (e.g., Mayer, 2010; Rosenshine, 
2006), but these have focused on skill levels found in high-income countries. With increased so- 
phistication in these countries, cognitive psychology applications have been viewed as somewhat 
simplistic. But for other populations they may be just right. To serve the poorest beneficiaries 
of the Education for All initiative, there is a need to better understand the available research 
and adapt its implications to the poorer populations. This article discusses the main concepts, 
needs, and potential opportunities for future research that may improve outcomes for low-income 
countries. 


COGNITIVE SCIENCE RESEARCH APPLICABLE TO EARLY-GRADE 
BASIC SKILLS 


The psychology concepts most useful for guiding instructional policies in low-income countries 
pertain to fairly basic functions: perceptual learning, working memory capacity, feedback, and 
the chunking process that packages related concepts together. Some uses of priority cognitive 
principles are outlined next. 

The most important area where learning falters early is reading. Although seemingly ele- 
mentary, the underlying cognitive neuroscience is definitely complex. Reading relies on a face 
recognition system. This has strict neurological requirements involving reactions in milliseconds, 
and these must be met before comprehension becomes possible. 

Students must first learn to detect and discriminate among multiple letter features. Recognition 
is greatly facilitated when features are few and when they conform to critical space and size 
recommendations (Pelli, Burns, Farell, & Moore-Page, 2006). The need to learn a few individual 
features and discriminate them from others is one reason why progressively presented synthetic 
phonics teaches basic reading efficiently in English and in other languages (see a review of various 
studies in Share, 2008). 

Practice results in rapid visual adaptation; some distinctions may only require a total of 2 hr of 
exposure (Hussain, Sekuler, & Bennett, 2011). Students in higher income countries rapidly learn 
skills such as discrimination of script features, but in low-income countries where prior exposure 


Cognitive psychology is the study of how people perceive, remember, think, speak, and solve problems. Psychologists 
have been conducting experiments on these topics since the 19th century. The term “cognitive science” often refers to 
interdisciplinary research on the mind and mental processes. The term “cognitive neuroscience” refers to research on 
brain states which directly correlate with mental states (Sun, 2008); for such research, neuroimaging methods are used. 
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to writing is limited, there is a need for explicit practice (Abadzi, 2006, 2008). However, textbooks 
are often scarce, and students get reading practice only from the blackboard. The blackboards 
may be small and scratched, and those sitting at some distance may not perceive letters as distinct 
entities or learn how to differentiate among them (Pelli & Farell, 1999). Typically the teacher 
writes a few words or a sentence and calls upon individual students to read before the rest of the 
class repeats in a chorus. After a few runs, however, children memorize the auditory sequence of 
the words and may repeat without knowing any of the letters. 

A visitor who changes the order of the words may find that only two or three students in 
the class can in fact decode them. Only the better off or brightest can learn with so little input; 
therefore, teachers tend to interact with just those few children able to retain large chunks easily. 
The rest of the class may be ignored, and they may gradually drop out of school altogether 
(Abadzi & Llambiri, 2011; Llambiri, 2004). Unfortunately, staff of donor agencies who visit 
such classrooms often have limited insight of these issues, so they may merely state that students 
should not memorize. 

To make sense of text, students must first read it fast enough to fit it in their working mem- 
ory. Practice helps chunk letters together and reduces reaction time (Miller, 1956). Eventually, 
the visual word form is activated; this is a part in the brain that facilitates the recognition of 
entire words (Shaywitz, 2003). The speed increases, sorting significant amounts of text into 
working memory. The relationship between speed and comprehension has been documented 
repeatedly (Breznitz, 1997; Laberge & Samuels, 1974, Snow, Burns, & Griffin, 1998), but with- 
out an understanding of working memory functions, educators may not see why it matters. As 
a result, some Grade | reading programs in low-income countries aim to teach comprehen- 
sion through text interpretation rather than practice for increased speed. Yet research supporting 
speed has been available for at least 15 years (Tan & Nicholson, 1997). The reading text- 
books that are procured tend to have large colorful pictures and small amounts of text. So, 
low-income children are expected to guess word spellings from pictures and rapidly automa- 
tize the large batches of letters. Even in well-resourced countries like Brazil, those who cannot 
may remain functionally illiterate for years while surrounded by beautiful books (OED, 2002). 
They may manage to decipher text only in Grade 4 and for years may read too slowly to 
make sense of sentences within the capacity of their working memory. Thus, they cannot an- 
swer many comprehension questions and cannot learn much information from texts (e.g., RTI, 
2010b). 

Insights about working memory lead to a realization that basic skills must be executed rapidly 
(Cooper & Sweller, 1987). The intertwined concepts of chunking and automaticity are crucial 
in understanding how to teach these students. Basic skills must be practiced to fluency, so that 
students can carry out the steps needed to process text, solve problems, or write ideas without 
thinking about them too much. Longer and complex chains of skills can only be developed after 
smaller chains have been formed through practice and use (Keele, 2005; Schmidt & Lee, 2005). 

The incremental nature of chunking has large-scale implications. Typically, governments 
and donors expect students to acquire flexible “21st-century skills” (World Bank, 2010), such 
as reading and understanding text quickly, calculating quickly, rapidly adapting at a job, and 
learning the necessary new content. However, years of learning and practice are needed to form 
and automatize the simpler components, and no easy shortcuts are known. Ignorance about the 
progressive size of information chunks leads to unrealistic expectations about students’ capacity 
to learn from complex methods and scant practice. 
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Because many students inevitably fail, governments must deal with extensive grade repetition. 
Automatic promotion is frequently recommended as a policy, but will promoted students master 
the prerequisites just by being exposed to more advanced topics? To be remembered, items must 
be laid into a bed of closely matching information (Katona, 1940, p. 162). Further, the path 
followed when information is encoded must also be used when information must be retrieved 
(Reisberg, 2009). Broadly, the research implies that kids promoted automatically to the next 
grade usually cannot learn what they missed without extra help. Remediation is needed, but 
tutoring is a private cost (Bray, 2007). Unfortunately, governments and donors rarely finance 
public remediation programs. The causal chain implies that remediation would be a priority 
strategy in lieu of automatic promotion. Structured programs during evening hours or vacations, 
potentially using older students, could enable more students to perform. 

Efficient teacher training could resolve many instructional problems. However, teachers in 
lower income countries often fail to improve classroom practices, despite massive investment 
in in-service training by donor agencies (Craig et al., 1998). Cognitive research would predict 
that teacher practices may not show long-term improvement after a dense 3-day course at a 
teacher training center. Content is encoded partly through environmental cues, so when teachers 
return to their classrooms, it may become a vague memory (state-dependent learning; Marian 
& Kaushanskaya, 2007). Also, long-term retention is based on distributive learning and review 
over a considerably long period (spaced learning; Pashler, Rohrer, & Cepeda, 2007). Thus when 
teachers go home and stop reviewing new concepts, work urgencies may overpower the memory 
of training course content. Trying to bring content into working memory while doing many other 
tasks in class may result in cognitive overload (Feldon, 2007). Efficient training of teachers 
who have limited education could include observational learning research. Behavioral sequences 
would be automatized, and they would more likely to be executed if shown explicitly. Visualization 
research also suggests benefits if teachers are asked to visualize the desired behaviors if they were 
carrying them out inside the classroom (Dowrick, 2011; Liu & Park, 2004; Wright & Smith, 
2009). Thus, video clips of the desired behaviors may prove a potent tool for the training of 
teachers with limited education. Preliminary tryouts have started in countries such as Liberia, and 
a methodology is under development. 

One obstacle to improvement is a limited insight on how people learn. Learning is often 
viewed as abstract and amorphous, without clear principles for its formation (e.g., Pigozzi, 2010). 
This is why the network-based structure of memory has considerable explanatory power. The 
semantic and propositional networks illustrate how pieces of information are linked (e.g., Collins 
& Quillian, 1969). Important factors that determine whether information will be retained and 
retrieved on various occasions include elaboration activities (a concept that partly overlaps with 
“active learning”) and the transformation of knowledge from superficial encoding to the “deep” 
structures that may lead to critical thinking (see Reisberg, 2009, for a review). Thus, knowledge 
can be shown to consist of relatively concrete (albeit very complex) nodes of information tied 
by links on the basis of some “rules.” Classroom activities can create, strengthen, modify, or 
erase the links. Donor and government staff can get a sense of how knowledge is constructed and 
modified? by gaining a better understanding of such knowledge. 


3Connectionist networks could be used to simulate instructional conditions in low-income classrooms and estimate 
educational outcomes, but this topic is beyond the scope of this article. 
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The network-based nature of memory also helps define “quality of education.” This term is 
used daily in donor and government planning documents, but its definition has been elusive. 
For example, an unpublished donor strategy document in 2009 defined quality as something 
that “unleashes the natural capacities of living beings to learn” while further stating that “good 
learning contexts enable quality learning, and bad learning contexts do not.” However, cognitive 
network concepts can again come to the educationists’ rescue. “LLow-quality” schools often teach 
a small amount of information, mainly memorized in series with limited connections to other 
knowledge; thus the students may retrieve the items only through direct questions about them. 
By contrast, “high-quality” schools teach more items, help attach them to multiple parts of the 
network, and give reasoning activities that result in new conclusions. The result is students with 
dense and connected knowledge networks, whose knowledge is retrieved through multiple paths 
and manifests quickly when needed. These students tend to score well in achievement tests and 
can perform in the labor market. With additional research, quality can be operationally defined 
and measured as a composite variable. 

The information-processing line of reasoning, therefore, has policy implications for govern- 
ments and donor agencies. It may focus vague and complex plans into a few relatively specific 
goals: speed and automaticity in basic skills, textbook acquisition for all students to maximize 
practice, measures to optimize use of instructional time, student remediation at public expense, 
and greater reliance on observational learning methods for teacher and supervisor training. Some 
implications seem to be common sense, but others, such as reading speed, may be counterintu- 
itive. Low-income governments have limited implementation capacity, but many receive strong 
donor support and are undertaking multiple initiatives. They are often willing to listen. 

The challenge, therefore, is to convince education staff to promote policies consistent with the 
information-processing chain of causality rather than policies promoted with little evidence. As 
the next section shows, this is not an easy undertaking. 


CHALLENGES IN USING COGNITIVE SCIENCE FOR EDUCATION 
POLICY GUIDANCE 


Most of the concepts used to clarify the problems of lower income classrooms are old. They were 
developed in the 1940s to 1980s and are found in undergraduate cognitive psychology textbooks. 
However, they have been hard to recognize and use because they do not stand out among the 
topics covered in those treatises. For example, chunking often gets only a passing mention. To 
some extent this is not surprising; cognitive psychology textbooks must be relevant to the realities 
of students who take the courses, so they tend to emphasize topics pertinent to more complex 
cognition. As a result, however, there is little organized content to teach university students about 
the essential cognitive concepts pertinent to millions of low-income students. Even graduates of 
psychology programs may have a limited understanding of these or know how to apply them. 

A related problem about some of these building-block variables (such as chunking or state- 
dependent learning) is limited research in educational paradigms (chunking has been studied 
more in procedural learning; Schmidt & Lee, 2005). Thus, relevant benchmarks are unknown. 
Automaticity in reading has been researched in conjunction with the visual word form activation 
(McCandliss, Cohen, & Dehaene, 2003; Shaywitz, 2003), but there is little research to specify 
how and when automaticity is attained for skills such as calculations. 
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Another obstacle in the use of cognitive research concerns the vocabulary available to express 
various concepts. The terminology has evolved over time without a clear direction, and some 
concepts seem minimally different from others. For example, there is short-term memory and 
working memory; generative learning (Soraci, Carlin, Toglia, Chechile, & Neuschatz, 2003) 
and Paulo Freire’s (1973) generative words; there are neural networks, associative networks, 
connectionist networks, memory nets and cognitive networks; and there is cognitive psychology 
versus Cognitive science and cognitive neuroscience. Furthermore, some cognitive concepts used 
often in education have no suitable names. For example, it is clear from research (e.g., Katona, 
1940) that to retain information, there should be preexisting closely matching items that would 
serve as “hooks.” A partially fitting term seems to be Vygotsky’s “zones of proximal development,” 
which could be adapted as “nodes of proximal encoding.” The overlapping and unclear terms 
confuse staff who have little knowledge of the discipline and diminish the chances that they will 
devote time to learn the material. 


THE CHALLENGES OF KNOWLEDGE TRANSFER 
TO THE EDUCATION SECTOR 


The limited learning outcomes in countries that receive donor aid raise pressures for account- 
ability. For example, an audit of the effectiveness of the British bilateral aid (Department for 
International Development, 2010) showed a limited impact of donor financing and recommended 
aid reductions. Some organizations, like the Center for Global Development, urge donors to stop 
financing mere inputs and expect results first. 

Departments that prepare students for international education work ought to respond to this 
crisis. The concepts and implications discussed in this article could be discussed more broadly 
among education scholars and partnerships could be sought with interested psychologists. How- 
ever, faculty of the two disciplines rarely collaborate (Ansari & Coch, 2006; Dempster, 1988; 
Metcalfe, 2006; Ronstadt & Yellin, 2010). So, changes to education faculty curricula have been 
limited. Several universities in the United States teach educational neuroscience, but the relevance 
of the research to low-income countries thus far has been limited. 

To some extent, philosophical orientation may raise barriers. Some education professors may 
espouse beliefs that run counter to cognitive research findings. For example, constructivists may 
expect that children will construct knowledge networks but give little thought to the conditions 
needed for this to happen (Kirschner, Sweller, & Clark, 2006). High-income countries place 
great emphasis on individualized attention, cultural particularities, and the special status of every 
person. The author has often heard statements that everyone learns in his or her own way, that 
there are many models of learning, and that universal guidelines for learning do not exist. Learning 
topics may still be considered “theories of learning,” a 1960s term that implies that the concepts 
are merely beliefs. Relevant research may even be considered deterministic or “Skinnerian.” As a 
result, the neurocognitive commonalities that could predict how to teach the very poor effectively 
may be found unappealing and could be ignored. 

One added difficulty is limited background knowledge that may help academics follow cog- 
nitive research, retain details, and interpret findings for students. The author has conducted many 
training events for education professionals of donors and governments for more than 10 years and 
has experienced the difficulties of teaching this material. Some participants seem rather baffled by 
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working memory and automaticity implications; they tend to rely on generic explanations from 
their experiences and to overlook inconsistencies. They may entertain the new concepts for a brief 
period and quickly return to their usual decision algorithms. Changing the decision algorithms 
of adults who have used different concepts for a long time is in itself a significant challenge that 
requires research from a cognitive and social standpoint (Shtulman & Valcarcel, 2012). 

The limited engagement by faculty ultimately penalizes the students who take their courses. 
Graduate students spend years studying and passing tests on the terminology and concepts used 
by the great educators of the past, such as John Dewey, Jean Piaget, or Lev Vygotsky, and learn 
to reason about learning through concepts of the early 20th century. The students may also 
have limited understanding of hypothesis formation and testing. Many ask for jobs, but during 
interviews with the author, some have stated that there is no relationship between halting reading 
and a lack of comprehension, or that whole-language textbooks in French are appropriate for 
countries like Burkina Faso. Others may confidently state that the best way to teach children 
is through play but have no explanation or knowledge of relevant research. Thus many recent 
PhDs of departments such as educational policy, adult education, or international education seem 
unqualified to make valid decisions on issues affecting millions of children in a country. 

What actions can be taken to help donor and government staff use the learning research 
pertinent to international education settings? First, the research and its implications must be 
taught through one or more courses that could be termed “cognitive neuroscience of basic skills.” 
These could be graduate courses aimed at future staff of donor and government agencies but also 
undergraduate courses aimed at future teachers. Some of the pertinent material can be found in the 
author’s book Efficient Learning for the Poor (Abadzi, 2006), but science evolves and updates are 
needed. Faculties of education must define the relevant content and closely monitor the evolving 
research. 

For staff already working in the education sector, persuasive and thoughtful training is needed, 
aimed at providing answers to the problems they face. One way to convince participants of the 
utility of learning research is to juxtapose the decisions made with and without it. Attractive 
publications and e-courses could be aimed at international audiences and at teacher trainers with 
modest levels of education. E-courses could include animated graphics and video clips from 
relevant classrooms. Concepts must be “popularized.” For example, memory functions can be 
depicted as a large bottle with a narrow neck in lieu of complex diagrams; chunking processes 
could be presented as railcars being connected into long chains, and cognitive networks could be 
depicted as LEGO blocks that can be manipulated. There may be trade-offs between precision 
and popularization, but the advantages could be considerable. 

There is also a need for extensive research into the learning issues encountered in low- 
income countries. The studies often cited to elucidate the education of the poor are used in a 
translational context, and various parameters must be verified. Other topics require study, such 
as the minimum number of instructional hours that governments must ensure for the acquisition 
of reading automaticity in various languages and scripts. In principle, funding can be available 
from bilateral agencies and foundations that specialize in the education of the poor. 

In conclusion, an information-processing framework may greatly help understand the reasons 
for low performance in low-income countries. It may also help determine which policies are 
likely to promote basic skills acquisition. Many of the concepts involved are not new, but they 
are relatively unknown and are sometimes difficult to communicate to the public. Adapting the 
terminology presents challenges, but these are worth pursuing. 
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Insights from educational theory and practice have reached their limits in low-resource en- 
vironments. Inspiration from common sense also seems to be waning. Surely Lev Vygotsky, 
John Dewey, Jean Piaget, or Paulo Freire would encourage the use of research that would benefit 
the poor. Quality of education and rights to education for all are really based on dense and 
well-connected networks of knowledge, with automatized basic skills. 
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With its broader employability to the issues of underperformance that may emerge in educational 
systems internationally, this empirical study redefines and expands Albert Hirschman’s theory of 
voice, exit, and loyalty within higher education. The article formulates a new education-embedded 
theoretical framework that explains reactionary behaviors of students in corrupt educational systems. 
The new corruption coping theory defines a set of coping mechanisms that students employ in reaction 
to failing institutions. Relying on the survey data collected from 762 students and interview-based 
data from 15 purposely sampled current students or recent graduates of the public higher education 
institutions in Bosnia and Herzegovina, the study reinterprets the voice mechanism that Hirschman 
sees as a political tool capable of bringing about change within underperforming institutions as, 
ironically, severely diminished in its power when observed within a corrupt environment. This 
research similarly differentiates amongst various types of exit and finds that Bosnian students often 
react in ways not predicted by Hirschman’s model, leading to the emergence of a novel corruption 
coping theory presented in this study. 


INTRODUCTION 


An interest in the topic of educational corruption has grown among both practitioners and aca- 
demics, promoting each to research corruption and its impact in the developing as well as the 
developed world. In recent decades, the World Bank and other international organizations have 
labeled corruption as one of the key barriers to the sustainable development of poor nations. 
This phenomenon weighs more heavily on the weaker states than the developed nations. In the 
latter, instances of corruption are often resolved and those responsible are sanctioned, if and when 
discovered. The analysis that follows narrows its focus to understanding students’ corruption cop- 
ing mechanisms in Bosnia and Herzegovina’s’ higher education, and adopts Heyneman’s (2004) 
definition of corruption as “the abuse of authority for personal as well as material gain” (p. 637). 

A derivative of broader research conducted on students’ perceptions of corruption, social 
mobility, EU-nionization,” and their coping mechanisms, this article presents select findings on 


'Bosnia and Herzegovina and Bosnia are used interchangeably. 

*The term EU-nionization was coined by Sabic-El-Rayess (2009) and “refers to a set of political, social, cultural, and 
educational forces that are tasked with synchronizing the family of European societies” (p. 427). 
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students coping with corruption in Bosnian higher education. The broader study applied concur- 
rent mixed methods, combining the binary logistic regression and content analysis as appropriate. 
A studied sample was drawn from a population of Bosnia’s students attending public faculties: 
The sample comprised 762 students from six public faculties and 15 interviewed participants 
who either recently graduated or are currently studying at the public faculties in Bosnia. 

Emphasizing students’ ability to cope with exposure to educational corruption, I employ 
and expand on Hirschman’s (1970) theory of voice, exit, and loyalty within organizations. By 
extending this theoretical framework into the educational milieu and contextualizing it within a 
corrupt setting, the study captures and systematically analyzes reactions of stakeholders to the 
failing organizations: in this case, reactions of Bosnia’s students to corrupt universities. According 
to Hirschman, there are two responses to the deteriorating performance: the firm’s clientele will 
either opt to “exit” their institutional relationship or “voice” their dissatisfaction with the failing 
organization (p. 4). Although claims of widespread corruption within Bosnia’s higher education 
are frequently made, little scholarly research analyzes how youth navigate through the corrupt 
educational milieus. This study, therefore, aims at formulating the corruption coping theory that 
models the behaviors of Bosnia’s youth who are encountering corruption. 


BACKGROUND: CORRUPTION IN BOSNIA AND HERZEGOVINA’S 
HIGHER EDUCATION 


Characterized by a complex governing structure and international dependency, postwar Bosnia 
continues to face a vast array of challenges, including broad societal corruption. According 
to Transparency International (2004), only 10% of the sampled population in the Federation? 
believes that corruption was significantly present in the prewar period, whereas 20% of the 
sample from the Serb Republic thinks that corruption was notably present during the same 
period. However, Transparency International indicated that, in 2002, a much higher proportion, 
85%, of the surveyed population in the Federation perceived corruption as significantly present 
in the postwar period, whereas more than 90% of those surveyed in the Serb Republic concurred. 
According to Chapman (2002), 38% of Bosnia’s students indicated that corruption was widespread 
among university professors. 

Over the years, Transparency International B&H has played a key role in evaluating the state 
of corruption within Bosnia’s universities. Using a representative sample of 500 students from 
the University of Sarajevo, Transparency International B&H found that 60.2% of the sample 
thought “that there is a great presence of corruption at the University [of Sarajevo]” (Knezevic, 
2005). Although Bosnia formed the Agency for Prevention and Coordination of the Fight Against 
Corruption, its head, Sead Lisak, has indicated that Bosnia is dealing with a broader societal 
problem as Bosnians “bribe even for a ‘good’ cemetery location” (MagazinPlus, 2011). Systemic 
corruption negatively impacts development, and its economic implications are significant in the 
arena of education as well, especially as they relate to educational costs and rates of return on 
higher education (Heyneman, 2010, 2011). 


3Dayton Peace Accord of 1995 split the country into two ethnic entities: The Federation of Bosnia and Herzegovina 
(51% of the territory) was given to Bosniaks (Bosnian Moslems) and Croats, whereas Serb Republic (49% of the territory) 


was given to Serbs. 
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METHODOLOGICAL APPROACH AND SAMPLING 
Mixed-Methods Approach 


The study used the concurrent mixed methods, collecting data both by surveying and interviewing 
students at Bosnia’s public faculties on their perceptions of corruption and their corruption 
coping mechanisms. Informed by a pragmatic worldview (Creswell, 2009), I draw on theoretical 
frameworks and analytical tools that optimize my research inquiry. In analyzing data collected 
via interviews, the broader study applied the content analysis by reducing detailed interview- 
based data to relevant themes and looking for concept repetition. The study also relied on 
survey-based data to uncover current trends in students’ behaviors and perceptions within corrupt 
higher education in Bosnia. Where pertinent, the broader research applied the binary logistic 
regression; however, the findings on corruption coping mechanisms presented herein did not 
warrant regression analysis and primarily rely on the interview-enhanced descriptive analytics 
extracted from the survey-based data. 


Sampling Process 


According to Bosnia and Herzegovina’s last prewar census of 1991, the country’s population 
totaled 4,377,033, of which 3.7%, or 122,967 individuals, were college educated. A politically 
contentious and first postwar census is currently ongoing, so the exact number of those attending 
and later graduating from both private and public colleges remains unknown as of October 2013. 
Even though the private universities and colleges often came up during the interviews with 
students, the focus of this research is on Bosnia’s public higher education as public institutions 
play a central role in educating new generations of Bosnians. 

There are eight public universities and an unknown number of unregulated and unaccredited 
private institutions of higher education in Bosnia. Each university consists of multiple faculties 
that operate fairly independently from the university itself. This study surveyed current students 
from six public faculties and interviewed either current students or recent graduates from a 
similarly diverse group of public faculties. The sample was collected only at the institutions that 
allowed their student populations to be surveyed. Of the approached faculties, two ultimately 
declined to participate in the survey process despite their initial commitment to engage. This 
could have impacted the survey findings, but such claim cannot be asserted in the absence of the 
actual data from those faculties. 

Of the surveyed sample, 366 students (48.0% of the sample) were Ist-year students, 251 
students (32.9% of the sample) were in the 2nd year of their studies, 89 students (11.7% of the 
sample) were 3rd-year students, 51 students (6.7% of the sample) were seniors, and five students 
(0.7% of the sample) were of unknown status. Further, 64% of the participants were female, 
2% did not indicate what their gender was, and the rest were male. The interview-based data 
were obtained via extreme-case sampling from the circle of individuals known to the researcher. 
Although representativeness of the studied student population was unattainable using purposeful 
sample selection, this research benefited from the in-depth data collected from those willing to 
speak openly on educational corruption. 
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THEORETICAL FRAMEWORK: EXIT, VOICE, AND LOYALTY 


A continuous exposure to the lack of merit associated with educational corruption produces 
complex behaviors and reactions among youth. Heyneman, Anderson, and Nuraliyeva (2007) 
wrote a seminal piece on the costs of educational corruption. In the process, they evaluated the 
perceptions of corruption in higher education in several countries of Central Asia and Europe, 
noting that students, at times, depicted corruption as a “norm” (p. 5). It is precisely when 
educational corruption becomes ingrained into the educational system that students’ behaviors 
change and reflect pressures of a corrupt setting. 

Drawing on Hirschman’s (1970) renowned work on exit, voice, and loyalty seems particularly 
valuable here. For Hirschman, institutions have temporary performance deviations that are “lapses 
from efficient, rational, law-abiding, virtuous, or otherwise functional behavior” (p. 4). This 
definition includes actions that may be deemed illegal, potentially allowing for the inclusion 
of corruption and other forms of outlawed business behavior (Hirschman, 1970). However, 
Hirschman did not elaborate on such forms of performance deviations and instead focused on 
quality and efficiency problems that, in his view, are “repairable lapses” (p. 4). For Hirschman, a 
perfect institution does not exist, and all institutional players may regress at some point in time, 
but mechanisms of a competitive market help them rebound. 

According to Hirschman (1970), customers of the businesses that begin to underperform 
react to such changes in one of the two ways. Exit is a reaction with economic implications as 
it correlates with the company’s declining revenues, unless demand is inelastic and the price 
increase can compensate for the revenues lost due to some customers’ exit (Hirschman, 1970). 
Hirschman saw voice as a political act that ranges from simple complaints to massive public 
protests. Revisiting Hirschman’s work, Klein (1980) looked into the notions of voice and exit, 
and noted that the model of reacting through exit or voice is not sufficiently elaborate. 

Specifically, Klein (1980) questioned Hirschman’s (1970) assumption of mutually exclusive 
exit and voice mechanisms and pointed out that human behavior tends to be characterized by 
political apathy rather than activism. In addition, Klein underscored that voice explains why con- 
sumers are dissatisfied, whereas exit—often in the form of a simple and abrupt departure—does 
not elaborate on why the customers may be leaving. For Hirschman, whether customers opt to 
exit or voice their displeasure is ultimately a function of the level of loyalty the customers feel 
toward the company: More loyal customers will complain but remain with the company, whereas 
others will exit. There is a possibility that dissatisfied consumers choose neither to exit nor to 
voice their views; for critical Klein, this is the situation Hirschman explained by resorting to 
the notion of loyalty. There may also be multiple transaction costs impacting decisions whether 
to exit; the area of transaction costs has been a focus of Williamson (1981), who believes that 
a long-term relationship with a business counterpart will, eventually, affect the structure of the 
business deal. 

In education, various factors clearly complicate the exit mechanisms or ways in which students 
choose to cope. Impact on price of schooling is an important consideration, as students may exit 
because of the increased cost of education. The motivating factor for exit then would not be 
educational corruption per se but an increase in the price of education due to bribery. When 
the price of a particular service increases due to a bribe, the poor are more likely to exit relative to 
the wealthier segments of the population. This phenomenon has been studied by Heller (2001): He 
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finds that providing financial aid to the poor may help overcome “the existing low-tuition and fees 
[that can] still act as a barrier to college enrollment” (p. 30). When contextualized to the Bosnian 
educational sector, the poorest students may be—similar to what Kaufmann, Montoriol-Garriga, 
and Recanatini (2008) saw occurring in the broader public sector—opting out of the corrupt 
educational system because of their inability to address continuous exposure to and demands of 
corruption. Others, however, may simply adjust to paying the higher price, especially if no viable 
alternatives exist. 

The public universities in developing countries often function without regard for the presence of 
private competition and operate under the presumption of perpetual existence. Herein, Hirschman 
(1970) argued, politically active students would prove beneficial to the system of public education, 
as opposed to those exiting. Of interest, Klein (1980) disputed Hirschman’s “assumption of 
producer responsiveness” (p. 417). For Klein, Hirschman disregarded the issue of “professional 
dominance” of various institutions, a trait that perpetuates institutional confidence (p. 417). 
Existing research on educational corruption rarely explores the intricate relationship of corruption 
and students’ reactionary mechanisms to systemic corrupt behaviors, which is why the next section 
employs Hirschman’s work to pursue the topic in depth and within the context of Bosnia’s higher 
education. 


EXIT, VOICE, AND LOYALTY REDEFINED: NEW CORRUPTION 
COPING MODEL 


Many insights shared in Hirschman’s (1970) work on exit and voice are invaluable in analyzing 
the Bosnian students’ various forms of exit from higher education, their intent to remain loyal 
to the corrupt system, and ways of voicing their dissatisfaction with educational corruption. 
Hirschman viewed customers as either “alert” or “inert,’ where complaints by “alert” customers 
give a firm an incentive to improve its performance, whereas the “inert” customers remain loyal 
and allow the firm to rectify any issues on its own (p. 24). An expectation that a firm’s failure 
leads to an immediate exit relies on an assumption that a firm’s client can easily find a comparable 
substitute; however, as Hirschman pointed out, the customer does not always exit and may give 
the failing firm “some latitude as both price-maker and quality-maker—and therefore, in the latter 
capacity, also as a quality-spoiler” (p. 21). In the educational sector, an individual may not be 
willing to transfer to another university or voice his or her dissatisfaction because the transfer 
may be costly or take too long. 


Internal Voice 


To better understand manifestations of voice in a corrupt setting, I utilize Hirschman’s (1970) 
framework of voice, exit, and loyalty as an interpretative foundation for a corruption coping model 
that has emerged in Bosnia’s higher education. Ironically, I reinterpret the voice mechanism 
as severely diminished in its power when situated in a corrupt environment. Within corrupt 
environments, students often remain passive, and if they voice their dissatisfaction, they do so 
less aggressively. In fact, types of voice mechanisms in a corrupt setting can be indicative of an 
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FIGURE 1 Theoretical model for forms of voice. (color figure available online). 


unhealthy environment in which students operate. Students may be discouraged by, for instance, 
a complaint that was dismissed without proper investigation and limited public protests, if any. 

As the primary data collected in the course of this research suggests, had it not been for the 
fear of repercussions, the students’ voice would likely be as powerful as Hirschman (1970) would 
expect it to be. To account for the challenges of a corrupt setting, however, this research devised 
a new and education-specific corruption coping model with a range of outcomes for the voice 
variable. Specifically, voice can happen in lieu of exit, post-exit, and prior to exit (Figure 1). 

When voice occurs in lieu of exit, I theorize it can be an official, public, or internal voice 
(Figure 1). The official voice refers to complaints shared with the official university bodies or 
professors. Whether students decide to complain to the official committees that may be tasked with 
addressing corruption concerns, uncorrupt professors, or corrupt professors ultimately impacts the 
form in which voice manifests itself: the /oud, moderate, or low voice, respectively. The magnitude 
of voice, dependent on its potential effectiveness in addressing the institutional deficiencies, is 
classified as loud, moderate, or low. Contacting the official bodies within the faculty can result 
in a loud voice, as such reaction is most likely to instigate an official inquiry into the student’s 
complaint and arguably result in a visible attempt to correct the deficient behavior. Although 
potential for change is greatest with a /oud voice, substantive change is not a certain outcome. 
If students fear potential consequences following their complaint, the /oud voice is an unlikely 
course of action. Students may also perceive the official committees as institutional posturing 
rather than effective anticorruption bodies. 

This study found that, out of 220 surveyed students, 28.7% of the surveyed sample that 
admitted they knew someone who had complained, 129 students or 58.6% of the subsample said 
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FIGURE 2 Level of student satisfaction after complaining. Source: Student surveys. (color figure available online). 


the friend who complained was “‘very dissatisfied” (Figure 2). Whereas 14.5% stated the friend 
was “somewhat dissatisfied” and 17.3% were “neither satisfied nor dissatisfied,” only 4.1% of 
the subsample were “very satisfied” with how the complaint was processed (Figure 2). Similarly, 
5.5% of the subsample. said that the level of satisfaction would best be depicted by “somewhat 
satisfied” (Figure 2). These findings evidence the lack of effective action resulting from students’ 
voicing of their dissatisfactions with the corrupt institutional framework. The analysis further 
found that 84.5% of respondents thought there were no committees to which one could voice a 
complaint, whereas only 11.3% of the surveyed pool was aware of the committees’ existence. 

Further confirming that the current environment impacts student behavior, a vast majority of 
students would be inclined, at least to some degree, to voice their dissatisfaction with corruption 
if they felt they could do so safely. A surprisingly high portion of the surveyed sample, 78.4%, 
stated that, if they could complain anonymously, they would: Specifically, 35.9% said they 
would “definitely” complain, 21.9% stated they would “probably” complain, and 20.6% said they 
would “maybe” complain (Figure 3). However, even if anonymity were secured, 8.1% would not 
complain, whereas 9.8% were not sure (Figure 3). 

Given the potentially unfavorable outcome of reporting corruption, students may be more 
inclined to express their dissatisfaction by speaking with the uncorrupt professors: a moderate 
voice. Here, the impact is likely moderate due to a limited potential for change relative to the 
loud voice: The power of the uncorrupt faculty over the corrupt professors is especially limited 
in settings where corruption is systemic. In Bosnia, Tanovic observes (Svevijesti.ba, 2008) that 
powerful professors are often unqualified yet control institutional access to outside talent. In 
other words, the likelihood of correction following communication between an uncorrupt faculty 
member and a student is even less than when the official recourse is taken; therefore, the power 
of this particular voice is moderate at best. 

As compared to other forms of voice, students’ /ow voice is least effective in changing 
corruption processes and largely serves as an emotional outlet for students rather than as a 
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FIGURE 3 Anonymous voice mechanisms. Source: Student surveys. (color figure available online). 


corrective and capable tool Hirschman (1970) defined. Voice is /ow when students limit their 
reaction by informally complaining to a corrupt professor or a powerless staff member, as they 
are both unlikely to take any action on behalf of the student. Direct complaining to a corrupt 
professor may in fact result in further repercussions for the student, rather than in a correction of 
the corrupt behavior that prompted the complaint. 

In addition to the official voice, there is a public voice that often correlates with a highly 
dissatisfied student body seeking a change (Figure 1). It is synonymous with the voice Hirschman 
(1970) saw as a political tool. In this corruption coping model, I classify public voice as a loud 
response because it has a greater potential to jolt the existing system and lead to a correction 
of the behavior in question. For instance, dissatisfied but active students may organize public 
protests—thereby manifesting their /oud voice—or work with media to openly address corruption. 
This classification of public voice as loud, however, does not mean that it will necessarily correct 
corrupt behaviors. Rather, I argue that, of all reactions a student or group may choose, public 
voice has the greatest potential to instigate a change. This is particularly the case when such 
protests start occurring on a regular basis as, over time, continuous public pressure can create 
sufficient momentum forcing the educational officials and political elites to address corruption, 
in part if not in its entirety. 

In lieu of exit, students could react with their internal voice, which I classify as a low voice 
(Figure 1). When it comes to the internal voice, the impact is minimal as this form of voice 
is limited to complaining to the family members and peers only. Although often widespread 
among youth, an internal voice is powerless within a corrupt institution and, more broadly, 
within a corrupt system. Such a minimally invasive form of voice can only secure continuation 
of corruption rather than disrupt the status quo. 

If voice post-exit occurs (Figure 1), it likely manifests itself either as public voice or internal 
voice, where public manifestations are typically loud; public voice occurs through media outlets, 
public protests, or even court proceedings. Klein (1980) also brought up the possibility of voice 
post-exit. Internal voice, however, is of low volume and low impact, as it relies on limited 
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FIGURE 4 Corruption coping mechanisms. Source: Student surveys. (color figure available online). 


conversations between students and their family members and peers. Exit post-voice can also 
occur, especially in those cases where the students’ concerns remain unaddressed, when a student 
has been intimidated through the educational process (i.e., repeatedly failed), or when students’ 
activism has led to the faculty pressure on a student (Figure 1). 

Corruption in Bosnia’s higher education is tolerated, and this tolerance manifests in the 
students’ fear of their faculty’s control over their futures. One of the study participants articulated, 
“Most of the complaints [relating to corruption] were never resolved within the university senate, 
court or the public. Some processes were started but were never resolved. I do not know a situation 
where a professor was left without a job because of corruption” (Interviewee 5C). Consequently, 
58.5% of the surveyed sample said they cope with corruption by staying passive: “keeping 
up with the required work” (Figure 4). They simply opt for status quo and no-exit-no-voice 
strategy. 

The next most frequent coping mechanism among surveyed students was the internal voice: 
44.8% chose “talking with family and friends” over other forms of voice (Figure 4). Fearful 
of repercussions, Bosnia’s youth opt for the internal voice: an ineffective tool in substantively 
changing their circumstances. Although students’ /ow and internal voice helps them express their 
frustrations with corruption, /ow voice fails to serve as the corrective tool Hirschman (1970) 
envisioned it to be. A study participant confirmed that stories of educational corruption are shared 
“when we [friends] sit down for coffee and then start talking about who bought what [exam]” 
(Interviewee 7C). 

Of interest, 23.8% of the surveyed participants chose to cope by looking for connections 
or giving bribes: Approximately one in five students in Bosnia’s public higher education was 
seeking connections or ways to bribe professors (Figure 4). This process fundamentally distorts 
the manner in which students understand and pursue academic and, arguably, professional success. 
Even the student union, an organization formed to represent the interests of students, rarely uses its 
voice effectively; instead, students look “after themselves without having a broader perspective” 
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(Interviewee 3C). When the student union acts, “it is usually for someone’s hidden agenda to be 
achieved through the union” (Interviewee 3C). 

The noted hesitance with disclosing one’s grievances in a more aggressive and visible form 
is correlated with a fear of repercussions. Although loud and moderate voices may lead to an 
inquiry into the student’s complaint, they also increase perceived likelihood for that student’s 
marginalization: “Students are talking but not loudly [emphasis added]. No one is courageous 
to speak up publicly” (Interviewee 1C). Consequently, Bosnian youth continue to opt for the 
acceptance of corruption and status quo. 


Mental Exit 


The broader research, of which this article is a part, shows that corruption exists in Bosnia’s higher 
education—with 88.7% of the surveyed participants recognizing corruption as present (Sabic- 
El-Rayess, 2011, 2013); yet the same students keep the universities in business by appearing to 
continue to function normally within universities they label as corrupt. The majority of those 
enrolled drop out of Bosnia’s faculties in the Ist year of their studies (Pitkanen, 2008). This 
study, however, is interested in understanding why those who stay remain indifferent and how 
they continue to cope with corruption in their faculties. Inertness in Bosnia, this research finds, 
is occurring due to the extent of educational corruption and the students’ realization that their 
individual exits and voices would not instigate a systemic change required to eradicate widely 
present educational corruption. In some cases, students are also inclined to stay put, as the 
transaction costs of transferring may be too high. 

Similar to this study’s earlier adaptation of Hirschman’s (1970) voice to a corrupt educa- 
tional setting, this corruption coping model contextualizes the notion of exit within the corrupt 
Bosnian educational system. For Hirschman, exit is replacing an existing institution with another 
competitor. Although assuming that a similar form of replacement will not always follow, this 
corruption coping exposition reflects on different types of exit from a particular institution. I 
propose a theoretical model that differentiates among various types of student exit specifically 
in response to educational corruption (Figure 5). First, students may physically remain in the 
same institution but mentally exit due to ongoing corruption they witness. They can make non- 
physical or mental exit in one of two ways. Some students may lose motivation to work hard 
and instead seek ways to get by with minimal effort. Others may opt for nonphysical exit by 
pursuing their degrees while exiting mentally from their corrupt surroundings. Hirschman be- 
lieved in customers’ loyalty to a particular firm; instead, this education-specific corruption coping 
model presumes that students remain within the same corrupt university not out of loyalty but due 
to the students’ awareness of corruption’s systemic nature and consequent disempowerment it 
induces. 

When looking at students’ exit, this study finds that only 6.8% of the surveyed students “al- 
ways” think about leaving their faculty, whereas only 3.8% think of it “almost always” (Figure 6). 
However, 27.9% “often” think of leaving, whereas 24.8% “rarely” think of leaving their faculty; 
at the same time, 8.9% of the surveyed students “almost never” think of leaving, whereas 25.8% 
“never” think of leaving their faculty (Figure 6). The finding that more than half of the surveyed 
sample think of exiting rarely, almost never, or never may be due to costly transfer, lack of 
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FIGURE 5 Theoretical model for forms of exit. (color figure available online). 
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FIGURE 6 Frequency of thinking about leaving faculty. Source: Surveyed students. (color figure available online). 
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FIGURE 7 Exit due to corruption. Source: Surveyed students. (color figure available online). 


transparent and affordable alternatives, and ultimately students’ internalization of their perceived 
inability to change the status quo. 

Mental exit suggests that most Bosnian students accept the conditions in which they operate. 
Despite the evident dissatisfaction with corruption, very few students are willing to leave the 
system as a result. They may feel as though other faculties are equally corrupt and the costs 
of transferring to another faculty clearly outweigh the benefits of the transfer. In other words, 
widespread corruption is a deterrent from transferring nationally. When asked if they would 
leave their faculty due to corruption, 59.1% of surveyed sample or 453 students said “no way,” 
whereas 29.2% or 224 said they would leave “partly” due to corruption (see Figure 7). In line 
with the study’s corruption coping framework, only 3.4% and 4.3% of the surveyed sample would 
consider leaving their faculties “mostly because of corruption” and “only because of corruption,” 
respectively (Figure 7). 

Looking through Hirschman’s (1970) lens, students remain seemingly loyal to the higher 
education system in Bosnia. Even though 26.2% of the surveyed students believe that they fail 
an exam despite demonstrating sufficient knowledge, only few think of leaving their faculties. 
Although this trend—in Hirschman’s jargon—signals loyalty, such behavior in actuality reaffirms 
the existence of mental exit. With the lack of alternatives, there exists a general tendency to 
mentally exit: a participant underlined that “all students are aware of it [corruption] but stubbornly 
choose not to pay attention to it” (Interviewee 1C). In December 2011, Helsinki Committee for 
Human Rights similarly confirmed that students in the Serb Republic are unwilling to react to 
cases of corruption, if and when they witness them (Srna, 2011). 

Students’ acceptance of the systemic corruption and their realization that the power of their 
exit is unlikely to produce a fundamental change allowed mental exit to emerge. Educational 
corruption has been accepted as a norm, and 38.5% of the surveyed sample believes that dealing 
with corruption is “neither difficult nor easy” (Figure 8). However, 29.4% and 16.3% of the 
surveyed students, respectively, believe that it is “difficult” or “very difficult” to cope with 
educational corruption (Figure 8). Even though corruption is widely accepted by many, others 
continue to have a difficult time dealing with corruption. Only 5.7% and 4.8% of the surveyed 
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FIGURE 8 Exit due to corruption. Source: Surveyed students. (color figure available online). 


sample was of the view that it is “easy” or “very easy” to cope with corruption, respectively 
(Figure 8). 

Aside from mental exit, students may also transfer nationally to another faculty, program, or 
university. Here, the locality of transfer is not of interest, but the rationale behind the transfer 
is of interest. For instance, 18.5% of the surveyed respondents do not wish to transfer because 
they view other schools as corrupt (Figure 9). Of the surveyed sample, 16.6% of students see the 
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FIGURE 9 Reasons for status quo. Source: Surveyed students. (color figure available online). 
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complex administrative processes as obstacles to transferring, whereas 13.6% of students blame 
transfer costs (Figure 9). In addition, 3.3% are discouraged by the paperwork, and almost half of 
the surveyed respondents (45.6%) said that two or more of the factors just discussed play a role 
in their decision not to transfer (Figure 9). In some instances, students may even wish to transfer 
to more corrupt institutions to graduate faster. This transfer can occur to another department 
within the same faculty or can extend to transferring to entirely different institutions and, in many 
cases, into ghost private institutions. Those who are in discord with corruption may transfer to 
less corrupt institutions domestically—be it within the same university or to another university 
(Figure 5). Others may go abroad to work or study, and some may exit higher education, both 
domestically and internationally, due to the pressures of educational corruption (Figure 5). 

Much of the research conducted on corruption often presumes that students are upset with 
the educational corruption, and this study does so to a certain extent. However, it is important 
to recognize the form of exit that is pursued by students who espouse educational corruption 
and purposefully look for the more corrupt colleges. In general, Bosnian students have a sense 
of which faculties and/or universities are more or less corrupt, and therefore know where they 
can obtain their degrees easily. Observing such a behavior enhanced the appeal of expanding 
and adapting Hirschman’s (1970) framework to corrupt educational systems; in the world of 
business, a customer would be unlikely to replace one inefficient company with a more inefficient 
organization. Such behavior may, however, be a driving force behind some students’ desire to 
exit and take further advantage of the ongoing corruption. 

In corrupt settings, students often develop a certain level of resilience concerning their repetitive 
exposures to educational corruption. Although one would expect that those in disagreement with 
the ongoing corruption within their faculties would look for alternatives, Bosnian students often 
stay put. Although it may seem counterintuitive, one of the study participants classified staying at 
the corrupt faculty as a form of resistance to the ongoing corruption (Interviewee 1C). Departing 
from the faculty would be a preferred student reaction by those who wish to eliminate competition 
from the capable and hard-working students: therefore, despite the presence of corruption, “the 
only way to fight against the uneducated is to educate oneself” (Interviewee 1C). 


CONCLUSION 


The main purpose of this research was to understand the impact of corruption in Bosnia’s higher 
education and develop a theoretical framework for the students’ corruption coping mechanisms. 
Hirschman’s (1970) work on the mechanisms of voice, exit, and loyalty informed creation of a 
new and education-specific reactionary model with a range of new outcomes both for the voice 
and exit mechanisms. The study hopes to have made a first step in providing a more complex 
interpretive framework for the reactionary mechanisms of the youth operating within corrupt 
educational systems. In the context of Bosnia, voice is disarmed from its potential for change, as 
it largely manifests itself through conversations with other students, friends, and family members. 
As to the exit mechanisms, the physical exit, be it to more or less corrupt institutions, does not 
occur frequently in Bosnia. Instead, it is the mental exit that dominates the educational setting. 
Most students, following realization that their individual reactions would not instigate a systemic 
change or that the transaction costs of transferring would be high, either lose motivation to work 
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hard or simply ignore broader corruption and push through the system as if the ongoing corruption 
in no way obstructs their paths. 

Bosnian students’ coping abilities have been impaired by extensive and systemic corruption. 
Students’ fear of authority not only deters students from realizing their full educational potential 
but also strips the country’s youth of its power to move Bosnia forward. Powerful signals in the 
form of massive exit and loud voice are absent from Bosnia and Herzegovina’s higher education, 
and this pioneering effort hopes that others will be intrigued by the study’s findings and move 
toward exploring, more closely, the coping mechanisms of students facing corruption throughout 
developing countries. 
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Corruption is at the core of weak governance. In the education sector, corruption is a threat to 
the quality of and access to education. Although the diagnosis is straightforward, effective reforms 
are more difficult to implement. The principles of good governance (transparency, participation, 
accountability, and integrity) provide us guidance, but innovative ways need to be found to fill these 
principles with life and create dynamics of change. We present a simple, efficient experience of 
introducing transparency and fostering participation and therefore accountability at the school level 
in Honduras. The transparency bulletin boards (murales de transparencia) are a homegrown response 
to the call for improving governance in a challenging environment. 


Corruption is at the core of weak governance. In the education sector, corruption is a threat to the 
access to and quality of services. Although the diagnosis seems straightforward, reliable empirical 
evidence and publically available data are often difficult to find and effective reforms are difficult 
to implement. Honduras is among the more challenged democracies in Latin America and has 
one of the lowest corruption perception indices in the region.’ Honduras is also severely limited 
in its ability to provide its citizens with quality public education services, among others. The 
challenges to improving the quality of education in Honduras are extremely complex. Decades 
of weak sector governance, politically thwarted reform initiatives, and high spending in a social, 
economic, and political context where corruption is endemic have resulted in little progress. 
Student achievement is among the lowest in the region. 
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The principles of good governance (transparency, participation, accountability, and integrity) 
provide us guidance to address some of the challenges at sector level, but innovative ways 
need to be found to fill these principles with life and create sustainable dynamics of change. 
Transparency bulletin boards (murales de transparencia) are a simple, efficient tool to introduce 
transparency and foster participation and, therefore, accountability at school level in Honduras. A 
simple bulletin board created by directors and teachers provides basic information for the school 
community. More important than the bulletin board itself and the information it provides are the 
dynamics of participation and accountability it creates. In many cases, this is the first time that 
parents actually see that the government is taking concrete steps to be accountable to them; this 
motivates questions and discussions among stakeholders in the education community at local and 
regional levels beyond the information provided. The bulletin boards are a homegrown response 
to the call for improving governance in a challenging, corrupt environment. 

This article presents the transparency bulletin boards in the context of a growing body of 
evidence on the effects of corruption on the education sector and a slowly emerging dynamic of 
positive change in the education sector in Honduras. The experience is new, beginning with a 
pilot in late 2010. The story, we hope, will serve other actors in similarly complex situations. 


THE PROBLEM OF CORRUPTION AND WEAK GOVERNANCE 
IN EDUCATION 


During the last 10 years, our understanding of how different types of corruption may undermine 
good governance in the education sector, and how this in turn may affect the sector’s outcome, 
has increased significantly. Huang (2008), for instance, found a significant negative effect of 
corruption on educational quality (composite index derived using each country’s average science 
and mathematics scores) and on educational quantity (school life-expectancy indicator developed 
by UNESCO) while controlling for other variables. 

How exactly does corruption affect educational quality and quantity? Corruption diverts scarce 
resources from intended purposes impeding adequate government spending on education. Mauro 
(1998) found a significant negative correlation between corruption and government expenditures 
on education—a finding that has recently been confirmed by Delavallade (2006), and De la 
Croix and Delavallade (2009). Also, corruption can impede effective government spending on 
education. Funds allocated to education may not reach their intended destination. An often-cited 
public expenditure tracking survey (PETS) carried out in 1995 in Uganda showed that only 20% 
of the allocated public funds actually reached schools. Part of the money was being captured by 
local elites (Reinikka & Svensson, 2004, 2005). Corruption also affects quality of education and 
access. For instance, Svensson (2005) found a negative relationship between corruption and the 
number of years spent in school, which could serve as a rough proxy to the level of primary and 
secondary education children get. Corruption need not involve money; stealing time also affects 
the quality of education. Ferraz, Finan, and Moreira (2012) empirically linked corruption in the 
education sector and at the municipal level more generally to lower educational outcomes, higher 
dropout, and failure rates among primary school students in Brazil. They also accounted for the 
difference between corruption and mismanagement of educational resources, which, as we show 
next, is a particularly relevant distinction in the case of transparency bulletin boards in Honduras. 
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If teachers do not show up in classrooms, or provide private classes for cash, this also affects 
both quality and equity of education (Patrinos & Kagia, 2007). There are legitimate reasons for 
teachers, as any professional, to be absent from work. Absenteeism and “ghost teachers,” however, 
are examples of corrupt behavior and practice. The World Bank’s 2004 World Development Report 
Making Services Work for Poor People generated groundbreaking new data from nationally 
representative surveys in six countries to better understand the extent and cause of teacher 
absence (World Bank, 2004; see also Chaudhury, Hammer, Kremer, Muralidharan, & Rogers, 
2006). As we discuss next, data on teacher absence in Honduras are not available. Teacher absence 
is, however, a national phenomenon, and the transparency bulletin board is a first step toward 
generating reliable information beginning at the school level. 

The links between corruption and gender inequality are particularly striking in the education 
sector. Case studies from different countries report gender-related violence and teachers abusing 
their authority to extort sexual favors. Gender-based corruption may also consist of students 
seeking to gain special treatment or better grades (see Bakari & Leach, 2009; Collins, 2009; 
Transparency International, 2007; U4, 2006). Heyneman (2004, 2009) pointed out another aspect. 
The education system plays a key role in guaranteeing fair access to the positions within a society 
based on merit and skills. If, in turn, positions are awarded according to kin, ethnicity, or political 
party, the system cannot guarantee that the selected person has the capacity to carry out a certain 
task in an effective and efficient way. 

Ultimately, Patrinos and Kagia (2007) reminded us that if children are exposed to corruption 
in schools, this may “undermine an entire generation’s core values regarding accountability, 
personal responsibility, and integrity” (p. 65). Further, an education system able to show and 
transmit the value of integrity may play a key role in reducing corruption in a country in the 
long term. Truex (2011), based on survey evidence from Nepal, found that education is a primary 
determinant of attitudes toward corruption. With increased levels of education come decreased 
tolerances for corruption, suggesting that improving access to education may be an important 
factor for reducing corruption. 

In short, there are many good reasons to invest in reducing corruption risks in education through 
improving governance. For instance, the results of the aforementioned Ugandan PETS, after being 
published in local newspapers through a government-led information campaign, triggered deeper 
reforms at local level in response to the growing public pressure.” A new PETS carried out in 
2002 showed that schools received more than 80% of the budget allocations in 2001 (Reinikka 
& Svensson, 2005). 

Our objective is to contribute to this literature and to share the experience of promoting 
transparency and participation in the education sector from Honduras. The next section provides 
an anticorruption and good governance framework based on the principles of transparency, 
participation, accountability, and integrity against which we analyze the case. The third section 
proceeds with a case study containing a description of the challenges the education sector faces 
in Honduras to get a grasp of the difficult context within which the transparency bulletin boards 
emerged as a first response to a complex social and political dynamic. By linking the experience 
to the framework from the second section, we explain how and why it works, highlighting the 


"Indeed, Hubbard (2007) explained that the reform dynamic was more complex than just the newspaper campaign. 
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underlying mechanisms and potential success factors. Finally, we extract some general lessons 
from this case, and ask the difficult question of sustainability. 


A BASIC FRAMEWORK FOR ANTICORRUPTION MEASURES 


Confronted with the task of minimizing corruption in education, the first step is to under- 
stand clearly the opportunities for corruption that may exist in the sector. Thanks to extensive 
research on the causes of corruption and a clearer understanding of different practices of cor- 
ruption, we now have various generic approaches that identify corruption “hot spots” in the 
sector. 

Heyneman (2004) considered what characteristics an uncorrupted system should have to then 
develop four different categories of educational corruption (see also Heyneman, 2009, pp. 2-4). 
First, the educational functions of the system can be corrupted, for example, the assessment of 
students, the graduation process, or the selection of students. Second, there are risks of corruption 
in the supply of goods and services, for example, the procurement of textbooks or of school meals, 
or the construction of schools. Third, agents of the education community may violate norms of 
professional behavior by, for instance, accepting gifts for grades, extorting sexual favors, not 
attending classes, or defrauding the education budget, for example, by pocketing salaries from 
different posts while working only in one.* Finally, the author points to possible abuses related 
to property and taxes. 

Hallak and Poisson (2005), in turn, disaggregated the education sector in different areas (school 
building, equipment and text books procurement, teacher appointment and management, teacher 
behavior, examinations and diploma, and information systems) in order to then examine how 
these areas might be prone to different types of corruption such as fraud, bribery, favoritism, 
embezzlement, or capture. Patrinos and Kagia (2007, p. 68) drew on this risk map by Hallak and 
Poisson for a value chain approach, dividing the value chain in two: between policymakers and 
providers (upstream), and between providers and beneficiaries (downstream). 

Such categories and risk maps are helpful to guide empirical work in specific contexts as they 
give hints on where to look. In the third section, we present an overview of studies of specific 
problems encountered in Honduras. 

With a clear understanding of the problems and their contextual specificity, it is possible to 
design interventions. As corruption breeds in opacity, and is essentially related to established 
power structures, the rationale behind anticorruption measures relies on both increasing infor- 
mation (i.e., transparency) and creating checks and balances (i.e., voice and participation; see 
also World Bank, 2004). Ultimately increasing information is an attempt to diminish asymmetric 
distribution of information between different actors, which enables corrupt practices. Providing 
information can foster accountability, which implies being held responsible for the tasks that 
have been entrusted to a person or organization. Transparency and participation are preconditions 
for effective accountability, as they may enable in- and outsiders to detect corrupt practices and 
other inefficiencies. Participation leads to a competition of interests, reducing the potential for 
capture of political or administrative decisions by a specific, narrow interest. Transparency, in 


3A social audit conducted in Intibucd, Honduras, has shed light on many of these practices (see Consejo Nacional de 


Anticorrupcién, 2009). 


90 F. BOEHM AND T. CAPRIO 


Transparency 
(Information) 
Participation 
(Competition of Interests) 
Integrity 
(Organizational and Individual) 


FIGURE 1 Basic framework for anticorruption measures. 
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turn, is a precondition for effective participation. For information to be used effectively, interested 
parties must be able to make the most out of it. This requires capacity, channels, processes, and 
ultimately interest and motivation to do so. 

Although transparency and participation are important, they are not sufficient to introduce 
real accountability and reduce corruption risks. Borrowing from basic game theoretical insights, 
there must be some negative feedback mechanism punishing undesired actions that violates 
established rules to trigger changes in behavior. If an announced sanction is not applied, the 
punishment will fail to deploy its corrective function, but also, and perhaps more important from 
a dynamic perspective, will undermine its intended preventive effects. The resulting impunity 
subverts even further the government’s credibility, and citizens may feel increasingly frustrated 
and cynical. That is, whenever efforts are invested in promoting transparency, it is important to 
think how detected corrupt behavior based on this improved information could be sanctioned. 
Sanctions do not only need to be formal ones, such as those provided by criminal or civil law, 
contractual sanctions, or administrative sanctions. Sanctions may also take the form of informal, 
social punishments, such as public naming and shaming in reports by civil society organizations 
or in the media, or an increased and openly expressed social discontent by the citizens. 

Finally, a range of anticorruption measures can be introduced within organizations, for instance 
the ministry or schools. The OECD (2009) referred to such measures as an integrity infrastruc- 
ture, as they aim at building organizational integrity. Such internal anticorruption measures can 
be rules based or values based, and can build integrity from top-down or bottom-up; typically 
we find a mixture of these approaches (see, e.g., Lange, 2008, or Lambsdorff, 2009). Standard 
measures include internal risk analysis, trainings, code of ethics, job rotation, internal whistle- 
blower procedures, random in-depth control, and so on. It is important to recognize the role of 
leadership (Rotberg, 2009) and intrinsic motivation (Schulze & Frank, 2003), as too much control 
may crowd out internal predisposition to be and to remain honest. 

Figure | summarizes the main pillars of an anticorruption framework for designing interven- 
tions, operating through two main channels: (a) transparency/participation and (b) integrity. The 
related measures are intended to reduce risks of corruption; pretending to eradicate corruption 
would be a quite heroic claim. 

However, caution is necessary here. The education sector is not insulated from broader gov- 
ernance issues and from the given institutional context. Budget transparency, for example, is a 
key aspect to foster a more responsible use of available resources but goes beyond the narrow 
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education sector. Also, if the justice system is inefficient or corrupt, it will imply problems for the 
governance situation in the education sector. For instance, as previously mentioned, if discovered 
corruption cases remain unpunished, the effect of preventive measures to reduce corruption risks 
introduced in the education sector are likely to be mitigated. Government failure to take appropri- 
ate punitive measures could inspire even more social discontent and distrust in public institutions 
than existed previously. Furthermore, it is pivotal to understand the political economy of a given 
country, and to ask questions such as “Who are the actors,” “What are their interests and possible 
hidden agendas,” and “What do power structures look like?” 

Benitez, Estache, and S¢greide (2010) argued that it is crucial to take into account the political 
economy of a country. Although the authors are interested in infrastructure sectors, the findings 
are easily applicable to other sectors. They argued that policy recommendations that are merely 
based on a sound knowledge of a given sector may fail precisely because broader issues were 
not considered. Private agendas behind policy choices, for instance, are the key to understanding 
why and how certain political decisions are taken or not taken. Actually, this approach helps us 
to explain many of the reform failures we witness in the education sector around the world. This 
importance of the specific context should teach us to be very cautious regarding the replicability 
of “best” or even “good” practices. 

This view coincides with Rodrik (2007), who argued in writing about the puzzle of develop- 
ment that basic principles of “good economics” actually are institution-free (e.g., property rights, 
incentives, sound finance, etc.). That is, it does not matter so much how these principles are 
attained, the important thing is that they are taken into account. The specific institutions con- 
tributing to achieve the aim of the principles, therefore, may well be (should be?) homegrown; 
this yields better results than trying to “transplant” good practices from one country to another. 

In essence, this idea of promoting local solutions also applies to the four principles of anticor- 
ruption just mentioned. It is important to reduce information asymmetries through transparency, 
but how to achieve it is open to many innovative ways and ideas. Also, creating competition 
between different interests involved through channels of voice and participation is important to 
reduce the risk of capture by any one of the interests; the same idea applies to fostering account- 
ability through different types of sanctions. Again, there is no panacea on how this might be 
translated into reality and results. 


THE TRANSPARENCY BULLETIN BOARDS IN HONDURAS 
The Context: The Education Sector in Honduras 


Honduras is one of the poorest and most unequal countries in Latin America. The country first 
held free elections in 1982 and still faces important challenges with respect to democracy. At 
the heart of the problem is weak governance. According to various international think tanks and 
indicators, Honduras is characterized by highly concentrated economic and political powers, a 
weak judiciary, and weak control over budget processes. The Worldwide Governance Indicators 
2010 from the World Bank capture the essence of the governance problems the country faces (see 


Figure 2).4 


4Ror more detailed information and other sources see, for instance, the reports from Freedomhouse (2007), USAID 
(2008), World Bank (2009), and the Bertelsmann Stiftung (2012). Boehm (2011) provided a general discussion and 
review of corruption and governance indicators for Honduras. : 


92 F. BOEHM AND T. CAPRIO 


HONDURAS 


Comparison with regional average (Latin America - lower bar) 





Voice and Accountability -— 


Political Stability 


Governnent Effectiveness | ———___ 


Regulatory Quality 


Rule of Law 


Control of Corruption 





8 25 56 75 168 
Country’s Percentile Rank (6-166) 


FIGURE 2 Worldwide Governance Indicators Honduras and regional average. CG) The Word Bank Group, Worldwide 
Governance Indicators (www.govindicators.org), Kaufmann, Kraay, & Mastuzzi (2010). Permission to reuse must be 
obtained by The World Bank Group, Worldwide Governance Indicators (color figure available online). 


The upper bars represent the Latin American average for each of the six indicators: voice 
and accountability, political stability, government effectiveness, regulatory quality, rule of law, 
and control of corruption. Even when taking into account the confidence interval (the black 
lines), Honduras scores lower than the regional average in every indicator. The perception of the 
political stability has suffered, of course, after the coup d état crisis in 2009; but Honduras scores 
particularly low in the indicators trying to capture “rule of law” and the “control of corruption.” 
The Corruption Perceptions Index confirms the World Bank Indicator on Control of Corruption. 
The most recent Corruption Perception Index 2012 ranks Honduras in the bottom quarter (rank 
133 of 176 countries with a score of 28). 

Some progress has been made in specific areas, however. For example, regulatory quality 
has improved, facilitating business and reducing both incentives and opportunities for lower 
level administrative corruption (see, e.g., the Enterprise Surveys carried out by the World Bank 
in 2006 and 2010).° The government recently announced a plan for fighting corruption in the 
administration over the period from 2011 to 2014, with particular emphasis on budget transparency 
and procurement. It is not, however, the first announcement of the government to fight corruption, 
and until now actual implementation has lagged behind rhetoric. 

The situation in the education sector is equally characterized by decades-long administrative 
abuses and simultaneously an institutional discourse of getting serious about fighting corruption 
and improving quality. In 2003, Honduras completed its Education for All Plan (EFA) and was one 
of the first countries to join the EFA-Fast Track Initiative with concrete indicators and strategies to 
improve access to and the quality of primary education in the country. Despite modest progress in 


‘This data for 2006 and 2010 can be found at the World Bank Enterprise Surveys website (http://www. 
enterprisesurveys.org/Data). 
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terms of access up until 2008, lack of governance, low-quality teaching and learning, and some of 
the highest teachers’ salaries in the region characterize the system. Teachers unions are extremely 
powerful and often politically motivated; union membership is mandatory. Furthermore, ghost 
teachers are common (see, e.g., Fontana, 2008),° and there is no system of teacher performance 
evaluation. Recent teacher pay raises were linked to such a system and the raises were granted 
without evaluations (PREAL, FEREMA, 2011). 

The 2007-2008 audit of posts and salaries undertaken by the High Court of Auditors (Tribunal 
Superior de Cuentas) and financed in part by the World Bank and national funds revealed over 
10,579 irregularities in the system at an estimated cost of Lps. 901, 456, 729 (or approximately 
US$47.7 million).’ The infractions included pay for more hours than allowed by law, extra merit- 
based pay for undocumented academic qualifications, and additional pay for working in rural 
areas, when in fact the teacher did not report to work where assigned. Although the authorities 
are following up on the findings, there is little public information about the process and actions 
taken after nearly 4 years. 

Classes are taught in three 4-hr shifts in Honduras: in the morning, afternoon, and evening. 
According to current legislation, teachers may hold up to three full-time teaching positions. 
There are no controls in place to objectively know whether teachers are actually in the classroom. 
Absences are high on any given day. Nation-level union leadership frequently calls strikes. A 
number of irregular practices are common, among them: teachers rent out one or more of their 
posts, collect the full salaries, and pay their substitute a lesser amount; teachers collect a salary 
and simply do not show up for work (they work other jobs simultaneously; some are working 
abroad, mostly in the United States); and some teachers even collect salaries for more than the 
three posts legally allowed. 

As there is no control over actual days of school or hours taught, there is also little control over 
who is teaching where. District Education Offices (Direccién Distrital de Educacioén [DDi]) are 
the first line authority of pedagogic and administrative supervision. However, these offices are 
understaffed as well as ineffectively staffed (that is to say, politically staffed rather than based on 
merit), underbudgeted, undertrained, and subject to the often overwhelming and uncoordinated 
demands of data collection by the Ministry of Education as well as bi- and multilateral aid 
agencies. 

Two official lists of teacher assignments—which are often contradictory—circulate in the 
system. The central human resources department in the capital, Tegucigalpa, keeps one list and 
the Department Education Office (Direccion Departamental de Educacién [DDE]) keeps the 
other. Neither of these two lists necessarily reflects reality. Although there is no doubt that there 
are, indeed, committed and excellent teachers in Honduras, in short, we do not know exactly how 
many teachers are on the books and how many teachers are actually in the classroom teaching.® 

For these reasons, a Public Expenditure Tracking Survey published by the World Bank and 
the Inter-American Development Bank in 2010 emphasizes human resource management in the 


Many press articles also refer to this situation (see, e.g., “Persisten Irregularidades,” 2011). 

TAudit reports are available at the Tribunal Superior de Cuentas website (http://www.tsc.gob.hn). September 26, 
2011. http://www.transformemoshonduras.com/CP/tsc.php. 

8A pplication of the Stallings method of classroom evaluation in 754 classrooms in 153 schools in Honduras in 2011 
revealed that teachers were involved in academic activity on average 64% of class time. Otherwise, they were either 
socializing in the hallways or doing other nonteaching tasks during class time. Sixty-four percent is 21 percentage points 
below the OECD standard for good practice (Secretaria de Educacién, 2011, p. 27) 


94 F. BOEHM AND T. CAPRIO 


sector. The findings are compelling. Twenty-seven percent of the teachers from the sample could 
not be found in the database of the Ministry of Education. Almost one fourth (23%) of the 
teachers who, according to the official list, supposedly were working in a school actually didn’t 
work there. According to the World Bank report, they should be counted officially as ghost 
teachers; the corresponding salaries are estimated to amount $US71 million (World Bank, 2010, 
Vol. I: 16). Overall, the annual loss due to teacher absenteeism is estimated to exceed $US19 
million; around 13.5 days are lost per teacher per year, mostly caused by strikes organized by the 
teacher unions (World Bank, 2010, Vol. I, p. 17). In sum, the World Bank Institutional Governance 
Review from 2009 underscores “the strong bargaining power of the teachers’ unions that oppose 
any attempt to reform in the sector” and “the insufficient accountability of all of the actors” as 
being at the core of the poor results of the education sector (World Bank, 2009, Vol. I, pp. 27-28). 


The Transparency Bulletin Boards Pilot 


In this context of increasing unbridled corruption and public frustration over the state of the public 
education system, the Ministry of Education Transparency Unit championed a pilot program 
designed to increase accountability at the school, regional, and national levels while promoting a 
culture of transparency and integrity at the community level. This initiative is a concrete response 
to legislation and policy documents that provide the legal and institutional framework to fight the 
endemic corruption that characterizes Honduras. 

The Transparency and Access to Public Information Law enacted in 2006 recognizes that 
transparency and accountability are cornerstones to “the construction of an authentic democracy” 
and calls for the inclusion of relevant content in the classroom. More recently, the Education Plan 
2010-2014 promotes the participation of civil society in the sector by implementing social audits 
and organizing information campaigns to establish a culture of accountability. During the design 
phase in 2010, the pilot project also benefited from the government-wide “Year of Transparency,” 
one among several high-level and visible campaigns undertaken by the newly elected post coup 
d’état administration. 

The pilot was designed around a simple instrument—a transparency bulletin board. The bulletin 
board is meant to correct for limited access to public information generated by the Ministry of 
Education and relevant to the quality of education and the proper functioning of the school. The 
idea was inspired by the now-famous example of posting budgets for school grants on the door 
of the school in Uganda (Hubbard, 2007), making important information about service provision 
available to clients. 

In terms of our framework described in the second section, the provision of information 
(transparency) is a necessary but, as pointed out by Hubbard (2007), not sufficient step toward 
strengthening accountability and limiting the scope for corruption. The premise was that creating a 
culture of accountability, despite the challenging odds in Honduras, could begin with providing the 
community with simple, easy-to-understand information. It was also important from the beginning 
to consider that more is required than just providing information. There must also be someone 
who can and will use this information, who asks questions and cares about how the education 
system works, thereby creating social pressure toward positive changes. The pilot, therefore, 
focused strongly on the process of implementation, fostering participation by involving teachers, 
parents, the administration, and local civil society. In this highly politicized and polemical sector 
in Honduras, the instrument also indirectly provides a platform for directors and teachers with 
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integrity to be recognized by their colleagues and communities as participating in a positive 
change dynamic. 

The initial concept was to post information on one of the key governance issues in Honduras 
that local education stakeholders could immediately understand and monitor: the management of 
teachers. The two official lists of teachers, along with an explanation of how to officially report 
teacher absences and other irregularities at the school level, were published. The added value of 
the exercise for the Ministry of Education is to identify administrative irregularities and correct 
them in the central database managed by the human resources department and the database at the 
departmental level, providing the inputs for a much needed house cleaning. 

The Department of La Paz was chosen based on its historically low educational indicators com- 
pared to the other 17 departments, previously established contact with civil society organizations, 
the commitment of the Department Director of Education to participate in the pilot process, and 
logistical considerations. The Department of La Paz is divided into 19 municipalities. There are 
687 schools registered in La Paz. The Director of the District Education Office of the municipality 
of Cabafias was the first local education authority to volunteer to participate in the initial pilot.’ 

The Ministry of Education Transparency Unit was responsible for the design and implementa- 
tion of the pilot initiative, with technical cooperation provided by the program PROEFA (Support 
to Basic Education in the context of EFA-FTI), a Ministry of Education and GIZ development 
measure, financed by the German Federal Ministry for Economic Cooperation and Development 
(BMZ). Alliances were formed with the Department Education Office (DDE), District Education 
Offices (DDi), municipalities, the La Paz Women’s Network (Red de Mujeres de La Paz), the La 
Paz Network of Transparency Commissions (Red de CCTs de La Paz), and parents associations. 


The Key Actors 


As mentioned, a key factor for understanding the initial positive dynamic surrounding the 
transparency bulletin boards is the focus on fostering multistakeholder participation. 


e Ministry of Education Transparency Unit. The Transparency and Access to Public Infor- 
mation Law (Decree Number 170-2006) was passed on December 30, 2006. Article 5 
stipulates that every public institution must name a public information official and assign 
that official an adequate budget to carry out his or her functions. The Transparency Unit in 
the Ministry of Education was established in 2008. Attorney Delmys Marlene Chavez was 
the first director appointed under Minister of Education Marlon Brevé Reyes (Liberal Party) 
and still serves in this position in the current National Party administration. As previously 
mentioned, the unit was responsible for designing and implementing the initiative. They 
achieved this by collaborating with their administrative counterparts at the departmental 
level, the Departmental Education Offices (DDE), each of which includes a “transparency 
focal person,” and with the District Education Office Directors. 

e School Directors and Teachers. School Directors (who are also teachers by profession and 
often also teach classes) and individual teachers were involved with the pilot from the 





21m Honduras, education administrators (Department and District Education Directors) and School Directors are 
teachers by profession and, by law, members of at least one teacher’s union. 
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beginning and are responsible for designing and providing updated information for the 
bulletin board. 

Citizens’ Transparency Commissions (CCT). Article 59-B (revised in 2010) of the Municipal 
Law legally established CCT.'° Each commission comprises at least five representatives of 
community organizations, unions, the business community, and other social organizations 
in the municipality. An assembly of representatives of community organizations, unions, 
private sector, and other social organizations elects the members, who serve for 3 years and 
can be reelected for one additional period. The main objective of the CCT is to implement 
social audits at the municipal level. A social audit is a process of holding public institutions 
accountable for the efficient and effective use of resources and delivery of services based on 
active citizen participation. In Honduras, social audits have become an increasingly popular 
means of insisting on transparency in public financial management at the decentralized level 
since 2002, with the Poverty Reduction Strategy Paper. The La Paz CCT Network, together 
with the La Paz Women’s Network (see next), monitored the implementation of the initiative 
and executed a social audit of working teachers in four municipalities (192 schools) using 
the information provided on the bulletin boards. 

La Paz Women’s Network. The La Paz Women’s Network was founded in September 2007 
in the municipality of La Paz and has approximately 200 members. The organization’s 
objective is to promote community development as well as provide training for its mem- 
bers. Activities emphasize topics such as women’s rights, social audits, and participatory 
budgeting at the municipal level. The network lobbies municipal governments to create and 
implement public policies in favour of women. As mentioned earlier, together with the La 
Paz CCT Network, the Women’s Network participated in a social audit of the information 
provided on the bulletin boards, thus creating a demand for the initiative. 

The Municipalities. Honduras is administratively divided into 298 municipalities. Each 
mayor (alcalde) is elected for a period of 4 years. Elections are held during the same 
electoral process as voting for president and congress. Based on Article 91 (revised under 
decree 143-2009) of the Municipal Law, for the fiscal year 2011, municipalities were 
transferred 9% of the total national budget, of which 1% is earmarked for children’s 
and youth projects and programs, whereas 2% is for projects and programs in the areas of 
economic development, social protection, and violence prevention against women. Thirteen 
percent of national funds are to be invested in operational and maintenance costs for social 
infrastructure (e.g., additional nurses and teachers salaries, school repairs or budgets for the 
CCT). During the pilot phase, municipal authorities participated by providing information 
and operating funds for CCT auditing activities.'! In the case of the municipality of Yarula, 
the mayor contracted an additional teacher based on the results of the exercise. 

Parents Associations. Article 107 of the Education law of 1966 first institutionally pro- 
moted parents associations. Although the law called for associations to be formed in all 
public and private schools 35 years ago, even today few parents associations are legally 
constituted, which gives them the ability to receive and manage public funds. Most ac- 
tivities relate to improving school infrastructure. Few associations are directly involved in 


'OLey de Municipalidades de Honduras, Art. 59 reforma 2010. 


'l’A|though the Honduran National Association of Municipalities did not play an active role in the initial design, we 


think that they can and will be a key player in the future development of the initiative at the national level. 
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other management aspects or in activities more directly related to improving the quality of 
teaching and learning in the school. However, a new, positive trend is slowly developing 
with the introduction of the transparency bulletin boards and with recent progress designing 
and implementing school development projects (Proyecto Educativo de Centro) and school 
network projects (Proyecto Educativo de Red). To date there is little systematic, formal 
support for this structure. 

Parents associations in the context of the World Bank supported Proyecto Hondureno 
de Educaci6n Comunitaria (PROHECO) have a more comprehensive management func- 
tion. PROHECO schools are community-managed schools introduced into the public 
system in Honduras in 1999 with the objective of improving access and governance 
at school level. Organizations called “community education associations” (asociaciones 
educativas comunitarias) administer these community schools. Asociaciones educativas 
comunitarias are nonprofit organizations, legally registered with the Ministry of Edu- 
cation, operated by parents and/or community leaders and charged with all aspects of 
school administration, including paying teachers. In 2001, also promoted by the World 
Bank, Congress created an additional structure aimed to promote community participa- 
tion in schools, the ADEL, or the Local Educational Development Association (asocia- 
ciones de desarrollo educativo local; CERCA, 2004). As public schools, the PROHE- 
CHO aes were invited to implement the bulletin boards as well, with rather mixed 
results. 


Transparency Bulletin Boards: Implementation, Use, and Monitoring 


In addition to posting the two lists of teachers, the transparency bulletin boards include a flow 
chart that explains how any citizen can file an official complaint or report corrupt practices (see 
Figure 3). 

The Transparency Unit does not keep records on the number of complaints received via this 
particular channel. Therefore, it is not currently possible to determine whether the transparency 
bulletin boards have contributed to an increase in official complaints. Furthermore, commu- 
nication between the Transparency Unit and the Human Resources Department (charged with 
following up administratively on complaints) is weak. In September 2011, then Minister of Edu- 
cation, Jose Alejandro Ventura, assigned a staff member from the Human Resources Department 
to follow up on findings and official complaints registered with the Transparency Unit. In March 
2012, Dr. Marlon Escoto replaced Minister Ventura and subsequently, the Human Resources 
Department underwent staffing changes as well. In April 2012, a new liaison to the Transparency 
Unit was assigned. Communication and coordination between these two organizational units 
within the same ministry remains challenged. 


124 randomized monitoring exercise in Intibuca and Santa Barbara showed that the majority of the schools in Santa 
Barbara without a bulletin board (40%) are PROHECO schools. As a reason the report puts forward weak communication 
between PROHECO school directors and DDi and DDE (Iraheta, 2011). Another randomized monitoring exercise in 
Lempira, Copan, and Ocotepeque found that from the sample of 125 visited schools, only five didn’t install the bulletin 
board, four of them being PROHECO schools (Lemus, 2011). 


98 F. BOEHM AND T. CAPRIO 


























COMPLAINT Ministry of Education - 
Minimum Transparency Unit Will communicate within 5 days 
Requirements Tel/Fax 2237-4312 about follow-up measures 


Email: transparencia@se.gob.hn 









+ Place & Date 
+ Personal 





























information 
Cana 
telephone) 
* Detailed, clear 
description of the Oniiak - : 
complaint oe District Education Office in your Investigates and takes appropriate 
+ Signature community legal measures 
+ Statement that this 
complaint has also 
been sent to the 













local CCT and the 
Transparency Unit 
of the Ministry of 

Education 


Follow-up by the complainant, provides necessary additional information 


| | 


Citizens Transparency Commission ‘The Transparency Unity informs the 
(CCT) in your community complainant at the end of the process 
about measures taken 











FIGURE 3 _ How to report corrupt practices. G) Ministry of Education Transparency Unit (translation by the authors). 
Reproduced by permission of Ministry of Education Transparency Unit. Permission to reuse must be obtained by the 
Ministry of Education Transparency Unit. 


The initiative enjoyed the direct support of then Minister of Education, José Alejandro Ventura, 
from the inception of the pilot.'* In February 2011, the Minister declared the installation of 
transparency bulletin boards mandatory at the preschool, basic, and secondary education levels 
in the whole country (officially 19,480 schools), beginning in La Paz (687 schools).!* In July 
2011, the Ministry of Education expanded the initial pilot to the five other departments in western 
Honduras: Intibuca, Lempira, Ocotopeque, Copan, and Santa Barbara (4,255 schools). In addition 
to local, municipal, and department-level CCT, nine other regional civil society organizations 
were actively involved in the monitoring process: Espacio Regional de Occidente, Asociacion 
de Organismos No Gubernamentales, Organizacion para el Desarrollo de Corquin Copan, Plan 
International in Copan and Santa Barbara, Comisionado Nacional de Derechos Humanos, Red 
de CCT de Occidente, Organismo Cristiano de Desarrollo Integral de Honduras, and Comision 
de Accién Social Menonita y Hermandad de Honduras. 

The implementation of the initial pilot in La Paz and the subsequent expansion to the other 
five departments followed essentially the same process. The Transparency Unit first approached 
the DDE and then the DDi. Workshops were held with key actors with the purpose of sharing 
information and distributing the lists. In the process of expanding the activity to the western 
departments, an additional set of workshops was designed specifically for civil society organiza- 
tions, in order to offer them the opportunity to become involved in monitoring at the school level 
and to encourage actual use of this information. 


'3The Minister himself was a powerful union leader for 14 years before joining the newly elected administration in 
2010. He was president of the second largest union PRICHMA (Primer Colegio Magisterial de Honduras) and instrumental 
in negotiating for the generous teachers benefits guaranteed in the controversial legislation of 1997 (Estatuto Docente). 

'4Secretaria de Educacién, Reporte General 25 de enero 2010. http://www.se.gob.hn/190.5.81.199/SEE/archivos 
descargables.php. Centros de Educacion Pre-basica were excepted from this policy. Centros de Educacion Pre-basica 
are managed by community volunteers. 
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Although the Transparency Unit is officially responsible for monitoring implementation and 
use of the instrument, it was clear from the outset that its Capacities to monitor the process alone 
are limited. Therefore, the unit has joined forces with civil society organizations to delegate parts 
of the monitoring activities, Volunteers fill out a simple format and submit their results in hard 
copy. The Transparency Unit collects the forms, inputs data manually, analyzes the findings, 
and passes them along to department and national-level authorities responsible for taking action. 
The unit then reports finding and irregularities to three users of the information: the department 
director (DDE, for corrective actions), the national Director of Human Resources—Teachers (to 
take corrective actions), and directly to the Minister of Education. 

The results of a monitoring exercise based on a sample of 355 schools in the western provinces 
(7% of total schools) at the end of 2011 revealed that 31% of schools in the sample presented 
lists with errors or discrepancies, 167 teachers were on the payroll for a specific school and did 
not teach there, and 232 teachers were in fact teaching at a specific school but were not officially 
on the payroll for that school. 


Scaling Up 


According to monitoring exercises carried out in November 2011, 4,459 bulletin boards 
have been installed in the six western departments with GIZ/BMZ support.'> The Transparency 
Unit formed an alliance with the National Anti-Corruption Commission (Comision Nacional de 
Anticorrupcién) and the World Bank for the scaling-up phase in October 2011. The program 
was extended to the other 12 departments where, by end of January 2012, 6,194 bulletin boards 
had been installed with support from the World Bank.'® The alliance with Comision Nacional de 
Anticorrupcion is intended to help strengthen civil society’s participation in these departments. 
The design calls for implementing the transparency bulletin boards in three municipalities per 
department (total of 36 municipalities and 4,813 schools) and posting the information on the 
website of the Ministry of Education.!’ 

One key change to the design of the intervention is that the data collected regarding irregu- 
larities will be posted directly on the website of the Ministry of Education by representatives of 
civil society organizations (password protected access). 

With activities in the western six departments supported by civil society organizations and new 
activities in 36 additional municipalities in the remaining 12 departments, there are still 8,827 
schools in 12 departments that have not received direct support to implement the new policy. 
Efforts to reach these schools are currently planned for 2012 with support from the World Bank. 

Scaling up does not only need to imply coverage. The boards can also be used as vehicles 
for other relevant information. In addition to posting the different lists of teachers assigned to 
each school, some school directors decided during the pilot phase to post information about 
school capitation grants (the matricula gratis program) and other programs (like the World Food 
Program school feeding program). This unexpected development not only reflects that some 
directors understand the utility of the bulletin boards as a school management instrument but 


'5Based on Iraheta (2011) and Lemus (2011). 
'6Information provided by the Transparency Unit, Ministry of Education (personal communication, January 26, 2012). 
"Ampliar y optimizar el proyecto Murales de Transparencia, Unidad de Transparencia, SE 2011. 
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also exemplifies how easy it is to expand and adapt the bulletin board to specific needs. As a 
consequence, the inclusion of this information on school grants was officially required in 2012. 

As previously mentioned, Dr. Marlon Escoto Valerio, who is simultaneously rector of the 
Universidad Nacional de Agricultura in the Department of Olancho, replaced Alejandro Ventura 
as Minister of Education in March 2012. Minister Escoto reissued the order to implement the 
instrument in all schools in June 2012!8 and in his first 6 months in office has taken a number of 
highly visible, high-impact anticorruption positions. 


Promoting Change—Providing Better Services 


Without an impact analysis, it is unfortunately not yet possible to quantify the concrete impact 
of the bulletin boards. However, they have inspired visible changes in attitudes and behavior on 
several different levels: within the Ministry of Education—within the Transparency Unit itself 
and between administrative levels of the Ministry, in the relationship between government and 
civil society actors, in civil society actors and their organizations, and among teachers. 

The initiative has been a learning experience for the Transparency Unit. The experience has 
resulted in a good example of better division of labor between administrative levels of the system, 
a general weakness in the education system as a whole. The unit staff at central level no longer 
see themselves as just receivers of information. They are now more involved with communicating 
what is “transparency,” and see themselves as actors in the system. Furthermore, although they are 
responsible for monitoring the implementation of the bulletin boards at school level, they quickly 
realized that physically accomplishing such a task was not feasible. Strengthening communication 
and clearly defining roles and responsibilities with the department and district education offices 
has been recognized as crucial to success. Formally, the department education offices should 
assign a focal person for transparency. This did not happen until the pilot was implemented. 

In addition to literally practicing administrative decentralization in a system where there is 
much discourse and little evidence of such reform, the Transparency Unit has also shown that 
state and civil society actors can combine forces to improve official lines of accountability. In 
the process, they have defined a new leadership role for their unit and have reached a new 
understanding of their role at the central level and delegated supervision and monitoring to 
department-level staff. 

The level of involvement of Honduran civil society in the education sector provoked by the 
transparency bulletin boards is new. International nongovernmental organizations have long sup- 
ported Honduras in health and education, particularly after the catastrophic hurricane “Mitch” 
in 1998. In the aftermath of hurricane “Mitch,” the number of Honduran nongovernmental orga- 
nizations multiplied rapidly, focusing primarily on reconstruction and social control of Poverty 
Reduction Strategy funds. An organized civil society response to corruption in the education 
sector, however, is recent. 

The role of parents is also in the process of being redefined in Honduras and the transparency 
bulletin boards have contributed in a small, concrete way to this new development. Parents have 
traditionally been called on to support public schools financially, to help with raffles, or to finance 
and organize festivities and cultural events. The social and economic power of teachers in rural 


'8Oficio No. 0075-SE-2012, June 6, 2012. 
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communities, where parents are less educated and often illiterate, is unsurprising. Although 
parent organizations are not new, they are becoming more popular and active at the national level. 
The transparency bulletin boards provided many parents with their first opportunity to become 
actively involved in controlling the services that schools provide. 

Contrary to expectations, the transparency bulletin boards have been met with little resistance 
from teachers. On the contrary, many teachers were made aware of the degree to which human 
resource mismanagement affected them personally—that unbeknownst to them, for instance, they 
were not officially registered where they were teaching. The activity was seen as an opportunity 
to initiate the administrative process to clear up their individual status—and name, in this sense, 
ensuring their individual integrity in the face of administrative chaos. Ministry-level actors quickly 
addressed initial resistance from teachers in La Paz. Some teachers saw the transparency bulletin 
boards as yet another initiative to pit parents against teachers and even to privatize education (a 
common argument used by the teachers’ unions to oppose reform), but the Transparency Unit 
staff from Tegucigalpa was able to play a mediating role and correct misinformation. 


LESSONS LEARNED AND REMAINING CHALLENGES 


Although it is certainly possible to conceive relevant and sophisticated solutions to the governance 
problems encountered in any country, they all must pass a reality check when it comes to 
actual implementation. A key message from this article is not to stop short at dreaming of 
what would be an optimal response to the challenges but to also think of what is feasible and 
meaningful for local actors. Regarding the chaos of the human resource management for teachers 
in Honduras, one might wish for an information technology—based information management 
system that automatically updates with each transfer of a teacher from one school to another and 
keeps track at central level of assignments in each school within the whole country. Both political 
consensus and resources for major and fast systemic change are scarce, however. Until the high- 
stakes political and financial decisions can be taken to overhaul the entire civil service human 
resources management system at the national level, pragmatic solutions are key to building 
a critical mass of public participation and strengthening the demand for accountability. The 
transparency bulletin boards are certainly not a magic bullet, but arguably a feasible practice 
adaptable to a challenging environment. Although not resolving the problem on its own, the 
bulletin boards should be viewed from a dynamic perspective. 

Linking back to our basic model, the transparency bulletin boards clearly focus on the 
“transparency-participation” channel aiming at improving accountability. Although being highly 
cost-efficient, they are able to promote broader participation and discussion among different 
stakeholders on the issues of transparency and governance. By achieving this they can contribute 
to the process of strengthening the demand for accountability, which in turn opens space for a 
public debate on problems, irregularities, and administrative challenges. Raising awareness of the 
problems of corruption and weak governance for the sector performance indirectly also touches 
the “integrity” channel in our model, as this concept is, in the end, at the core of the intervention.” 
Clearly, parallel efforts to reduce corruption risks and improve governance structures have to be 


\°1t is worth noting anecdotally that the issue of integrity was also relevant in discussions and work with civil society 
organizations. In a context where “yellow journalism” is the norm, it was not only important but also necessary to stress 
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made on all levels of the education sector administration, and the initiative, like an eye-opener, 
has contributed to facilitating deeper governance reforms. Currently, the ministry is undergoing a 
process of developing its own internal anticorruption strategy, linking it to broader management 
reforms, responding both to an increased pressure by the government toward such reforms and 
the increased demand for accountability by citizens. 

A further important contribution relates to sustaining political will. Together with an increas- 
ingly critical population and an active and organized civil society that is tired of the pervasive 
problem of corruption in their society, measures such as the transparency bulletin boards may 
raise the costs for any government trying to reverse initiated reforms. This growing demand is 
of pivotal importance for making it politically attractive to foster governance reforms and anti- 
corruption measures; a clear symptom of the general discontent in Honduras is the newly formed 
Anticorruption Party with founder Salvador Nasralla running for president.” 

Moreover, the transparency bulletin boards should be seen in the context of various other 
initiatives to more actively involve local government and parents in quality control at the school 
level (e.g., participatory school planning, Proyecto Educativo de Centro). Also, in March 2011 
the government of Honduras passed a new education law promoting community participation 
and quality education (Ley de Fortalecimiento a la Educacién Publica y a la Participacién 
Comunitaria). This law was viewed as a first step to revising the general education law dating 
back to 1966. The “participation law” calls for the creation of municipal educational development 
committees (Comites Municipales de Desarrollo Educativo), the main purpose of which is to 
exercise social control over the education system at the municipal level. The Comites Municipales 
de Desarrollo Educativo are expected to monitor school capitation grants, the school feeding 
program, the conditional cash transfer program “Bono de 10 mil,” and teacher assistance. All of 
this information is already included in, or can easily be added to, the transparency bulletin boards 
and can thereby help in promoting the nationwide implementation of the new law. 

Sustainability of the measure presents two man challenges. First, 2012 will be a critical 
year for the initiative, when the Ministry of Education is expected to make staff and financial 
resources available to ensure 100% compliance and follow-up. PROEFA investment was minimal 
(approximately $US9,000 not including staff time). The World Bank support was valued at 
approximately $US38,000 (August-December 2011). In a fiscal environment where 80% of the 
education budget for basic education is spent on teachers’ salaries, few resources are available 
for even the most basic teaching and learning activities and other operational measures. Together 
with the challenge of participant motivation and momentum, the transparency bulletin boards 
and the Transparency Unit will need high-level political support as well as operational funds to 
ensure that the initiative is taken to scale to the extent that all schools, in 298 municipalities, use 
the method to generate change. 

The more crucial challenge is related to the response, or lack of response, of the government to 
the identified irregularities. The momentum of the positive change dynamic can be thwarted if the 
government does not follow up on irregularities and complaints. The process of implementation 
of the transparency bulletin boards has raised expectations; parents and civil society organizations 
that denounced corrupt practices or participated in social audits of the bulletin boards want to see 


the importance of documenting objective evidence and promoting dialogue in the interest of finding constructive solutions 
to common problems, instead of allowing the initiative to be co-opted in the service personal agendas and vendettas. 
°http://www.salvadornasralla.com/ 
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the infractions sanctioned and the irregularities resolved. If the involved parties do not see any 
real progress in the near future, this may further augment the disillusionment with state actors in 
a country where the confidence in democracy is already weak. 

We are confident, however, that despite these challenges, the current social environment and 
increasing public dissatisfaction with the quality of Honduran public schools, the recent legal 
reforms in the education sector and the anticorruption agenda of the current government combine 
to make for the right conditions for planting the seeds for long-term systemic change. 


AUTHOR BIOS 


Frédéric Boehm holds a doctorate in economics from the Technische Universitat Berlin in Ger- 
many. Amongst other, he is currently course expert at the U4 Anti-corruption Resource Centre, 
and professor in political science at Universidad Auténoma del Caribe in Barranquilla, Colombia. 
His teaching, consultancy work, and research focuses on institutional economics and game theory 
applied to corruption, governance, and institutional reforms. 


Temby Caprio, PhD, is Team Leader of the GIZ Basic Education Program for Afghanistan and 
German Education Sector Coordinator in Afghanistan, based in Kabul. From 2008 to 2012 she 
was Team Leader of the GIZ education program in Honduras, PROEFA. As a Peace Corps 
Volunteer in Cape Verde from 200-2002, Temby was a Teacher Trainer. Since then, she has 
worked for the GIZ as an education advisor on various projects for the last 10 years. 


REFERENCES 


Bakari, S., & Leach, F. (2009). I invited her to my office: Normalising sexual violence in a Nigerian college of education. 
In S. P. Heyneman (Ed.), Buying your way into heaven. Education and corruption in international perspective (pp. 
9-22). Rotterdam, the Netherlands: Sense. 

Benitez, D., Estache, A., & Sgreide, T. (2010). Dealing with politics for money and power in infrastructure (ECARES 
Working Paper 2010- 031). Université Libre de Bruxelles, Brussels, Belgium. 

Bertelsmann Stiftung, BTI 2012. Honduras Country Report. Gutersloh, Bertelsmann Stiftung. 

Boehm, F. (2011). Indicadores de Corrupcion — {Qué reflejan y qué omiten? El Caso de Honduras. In Consejo Nacional 
Anticorrupcién [Corruption Indicators — What do they reflect, what to they leave out? The Case of Honduras] 
(Ed.), Informe Nacional de Transparencia. Tegucigalpa, Consejo Nacional Anticorrupcion. Available from http: 
//www.cna.hn/node/49. 

CERCA Compromiso Civico para la Reforma de la Educaci6n en Centroamérica. (2004). Estudio cualitativo sobre 
la participaci6n ciudadana en el mejoramiento de la calidad de la educacion en cinco patses latinoamericanos 
Resultado de tres casos en Honduras (Qualitative study of citizen participation in improving the quality of education 
in five Latin American countries. Results from three case studies in Honduras]. Washington, DC: USAID/AED. 

Chaudhury, N., Hammer, J., Kremer, M., Muralidharan, K., & Rogers, F. (2006). Missing in action: Teacher and health 
worker absence in developing countries. Journal of Economic Perspectives, 20, 91-116. 

Collins, J. M. (2009). When schools fail to protect girls: School-related gender-based sexual violence in sub-Sahara 
Africa. In S. P, Heyneman (Ed.), Buying your way into heaven, Education and corruption in international perspective 
(pp. 23-50). Rotterdam, the Netherlands: Sense. 

Consejo Nacional Anticorrupcion. (2009). Resultados de la veeduria social al sistema educativo de Intibuca [Results of 
social oversight to the educational system of Intibuca]. /nforme ‘De Cara a la Ciudadania’ No. 10. Tegucigalpa. 
Retrieved from http://www.cna.hn/uploads/files/De_cara_a_la_ciudadania_10.pdf 


\ 


104 F. BOEHM AND T. CAPRIO 


De la Croix, D., & Delavallade, C. (2009). Growth, public investment and corruption with failing institutions. Economics 
of Governance, 10, 187-219. 

Delavallade, C. (2006). Corruption and distribution of public spending in developing countries. Journal of Economics 
and Finance, 30, 222-239. 

Enterprise Surveys. (2006). Washington, DC: The World Bank. 

Enterprise Surveys. (2010). The World Bank, Washington, DC: The World Bank. 

Ferraz, C., Finan, F., & Moreira, D, (2012). Corrupting learning: Evidence from missing federal education funds in Brazil 
(Working Paper No. 18150). Cambridge, MA: National Bureau of Economic Research. 

Fontana, A. (2008). Teachers and taxis: Corruption in the education sector in Honduras (U4 Brief 2008:16). Bergen, 
Norway, Chr. Michelsen Institute. 

Freedomhouse. (2007). Countries at the cross-roads 2007 (Country report). Washington, DC: Honduras Freedomhouse. 

Hallak, J., & Poisson, M. (2005). Ethics and corruption in education: An overview. Journal of Education for International 
Development, 1, 1-16. 

Heyneman, S. P. (2004). Education and corruption. /nternational Journal of Educational Development, 24, 637- 
648. 

Heyneman, S. P. (Ed.). (2009). Buying your way into heaven. Education and corruption in international perspective. 
Rotterdam, the Netherlands: Sense. 

Huang, F. L. (2008). Corruption and educational outcomes: Two steps forward, one step back. /nternational Journal of 
Education Policy & Leadership, 3(9), 1-10. 

Hubbard, P. (2007, December). Putting the power of transparency in context: Information's role in reducing corruption 
in Uganda's education sector (Working Paper No. 136). Center for Global Development. 

Iraheta, M. A. (2011). Control aleatorio de murales de transparencia en los centros educativos de los departamentos 
Intibucd y Santa Barbara [Random control of the transparency bulletin boards in the schools of the departments of 
Santa Barbara and Intibuca). Secretaria de Educacio6n/PROEFA, Santa Rosa de Copan. 

Kaufmann, D., Kraay, A., & Mastruzzi, M. (2010). The Worldwide Governance Indicators: Methodology and analytical 
issues. (World Bank Policy Research Working Paper No. 5430) 

Lambsdorff, J. G. (2009). The organization of anticorruption—Getting incentives right. In R. I. Rotberg (Ed.), Corruption, 
global security, and world order (pp. 389-415). Washington, DC: Harvard Kennedy School and the Brookings 
Institution Press. 

Lange, D. (2008). A multidimensional conceptualization of organizational corruption control. Academy of Management 
Review, 33, 710-729. 

Lemus, R. O. (2011). /mplementaci6n de murales de transparencia en los centros educativos departamentos Lempira, 
Copan y Ocotepeque {Implementation of the transparency bulletin boards in the schools of the departments of Lempira, 
Copan, and Ocotepeque}. Secretaria de Educaci6n/PROEFA, Santa Rosa de Copan. 

Mauro, P. (1998). Corruption and the composition of government expenditure. Journal of Public Economics, 69, 263- 
Dios 

Organisation for Economic Co-operation and Development. (2009). Towards a sound integrity framework: instruments, 
processes, structures and conditions for implementation (OECD GOV/PGC/GF [2009] 1). Paris, France: Author. 

Patrinos, H. A., & Kagia, R. (2007). Maximising the performance of education systems. The case of teacher absen- 
teeism. In E. Campos & S. Pradhan (Eds.), The many faces of corruption (pp. 63-87). Washington, DC: The World 
Bank. 

Persisten irregularidades en pago de maestros: Ministro de Finanzas asegura que 175 maestros muertos cobren salaries 
[Irregularities in teacher payments persist: Finance Minister says that 175 dead teachers charge salaries]. (2011, 
August 17). El Heraldo. 

PREAL, FEREMA. (2011). Educacion: Un desafio impostergable. Informe de Progreso Educativo 2010. [Education: An 
urgent challenge. 2010 Report on Educational Progress]. Washington D.C.: Programa de Promocién de la Reforma 
Educativa en América Latina y el Caribe (PREAL), and Tegucigalpa: Fundacion para la Educacién Ricardo Ernesto 
Maduro Andreu (FEREMA). Retrieved from http://www.thedialogue.org/PublicationFiles/HondurasRC2010-Final. 
pdf 

Reinikka, R., & Svensson, J. (2004). Local capture: Evidence from a central government transfer program in Uganda. 
Quarterly Journal of Economics, 119, 679-705. 

Reinikka, R., & Svensson, J. (2005). Fighting corruption to improve schooling: Evidence from a newspaper campaign in 
Uganda. Journal of the European Economic Association, 3, 259-267. 


FOSTERING GOOD GOVERNANCE AT SCHOOL LEVEL 105 


Rodrik, D. (2007). One economics, many recipes. Princeton, NJ: Princeton University Press. 

Rotberg, R. I. (2009), Leadership alters corrupt behavior, In R. I. Rotberg (Ed.), Corruption, global security, and world 
order (pp. 341-358). Washington, DC: Harvard Kennedy School and the Brookings Institution Press. 

Schulze, G., & Frank, B. (2003). Deterrence versus intrinsic motivation: Experimental evidence on the determinants of 
corruptibility. Economics of Governance, 4, 143-160. 

Secretaria de Educacion. (2011). Reporte de la Administracion del Tiempo en el Aula de Clases, Evidencia de Honduras 
utilizando el Método Stallings [Report on the management of time in the classroom, evidence from Honduras using 
the stallings method]. Washington, DC: Tegucigalpa. 

Svensson, J. (2005). Eight questions about corruption. Journal of Economic Perspectives, 19(3), 19-42. 

Transparency International. (2007). Corruption in the education sector (Working Paper No. 4/2007). Berlin, Germany: 
Author. 

Truex, R. (2011). Corruption, attitudes, and education: Survey evidence from Nepal. World Development, 39, 1133-1142. 

U4. (2006). Corruption in the education sector. U4 Anti-corruption Resource Centre Issue 4: 2006. Bergen, Norway: Chr. 
Michelsen Institute. 

USAID. (2008). Honduras corruption assessment report. Washington, DC; Author. 

World Bank. (2004). Making services work for poor people (World Development Report). Washington DC: Author. 

World Bank. (2009). /nstitutional governance review: Performance accountability in Honduras. Washington, DC: World 
Bank. 


World Bank. (2010). Public expenditure tracking and service delivery survey—Education and health in Honduras (Vols. 
1 and 2). Washington, DC: Author. 


PEABODY JOURNAL OF EDUCATION, 89: 106-122, 2014 g Routled e 
Copyright © Taylor & Francis Group, LLC 
ISSN: 0161-956X print / 1532-7930 online 
DOT: 10.1080/0161956X.2014.862476 


Taylor & Francis Group 


An Attempt to Reinterpret Student Learning Outcomes: 
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Attracting significant attention from the media and policymakers, international student assessments 
are serving as pressures on and challenges for individual countries. Even so, paying attention only 
to publicly released rankings in international student assessments may lead to superficial interpre- 
tations of the assessment results. Simply equating the rankings with educational outcomes can be a 
questionable perspective because the assessment results can be interpreted differently. In this context, 
this article has explored student achievement in the Programme for International Student Assessment 
(PISA) tests in comparison with civic knowledge achievement in the IEA Civic Education Study 
across 17 countries. This article has also evaluated efficiency in educational processes by assessing 
outputs (student performance in PISA) against inputs (educational spending and learning time) di- 
rectly used in the education system. These two approaches clarify that publicly released rankings in 
PISA change to some extent when the rankings are reevaluated by taking other factors into consider- 
ation. This implication should be allowed for in interpreting the results of international assessment 
and the relative rankings of participating countries. 


INTRODUCTION 


In today’s knowledge-based society, educational outcomes, generally measured in student aca- 
demic achievement, are primary concerns of the public at large, including policymakers involved 
in educational decision making. The public’s attention to educational outcomes has moved from 
the national to the international stage, in particular because of the appearance of cross-national 
assessment of student achievement such as the Programme for International Student Assessment 
(PISA) and the International Association for the Evaluation of Educational Achievement (IEA) 
studies. This transition has generated a growing fascination with comparisons of student learning 
achievement, as it is believed that cross-national assessments present the aspects of quality in ed- 
ucation in a cross-national comparative perspective, with a focus on student learning (Kellaghan, 
2006). Cross-national student assessments thus serve to internationally benchmark state perfor- 
mance standards (Phillips & Jiang, 2011), especially due to the fact that they provide the relative 
rankings in student achievement of participating countries. For instance, Finland is regarded as 
the best reference country. Linnakyla (2006) pointed out that international attention exploring the 
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“secret of Finnish success” has raised interest levels in the high performance of Finnish students 
in PISA. Furthermore, cross-national student assessments have been used as the primary metric 
for assessing the relative qualities (e.g., strengths and weaknesses) of national education systems 
(Bettinger & Slonim, 2005). 

However, international assessment results constantly precipitate heated discussion and debates 
among researchers, a variety of interest groups, and policymakers (Linnakyla, 2006). In under- 
standing these discussions and debates, we need to pay attention to two important issues. The 
first is whether the relative quality of education should be measured solely in terms of cogni- 
tive achievement (i.e., test scores) on such core subjects as mathematics, science, and reading. 
Cognitive achievement on the core subjects is generally harnessed as an indicator of educa- 
tion outcomes because of some distinctive characteristics such as relatively uncomplicated data 
collection, quantitative measurability and analysis, and its relationship with national economic 
performance. However, previous studies suggest that test scores are not the only means to measure 
student learning outcomes (e.g., Bowles, Gintis, & Osborne, 2001; Heckman, 2000). Specifically, 
Heckman (2000) argued that test scores used to assess cognition and academic ability produce the 
biased evaluation of human capital formation while excluding social adaptability as an academic 
outcome. It is axiomatic that education serves different ends, that is, schools are producing other 
outcomes besides cognitive achievement. Civic purpose of education, such as nurturing students 
to be responsible citizens and to participate in the public life of a democracy, is considered 
another essential component of education (Branson, 1998; Hinde, 2008; Homana, Barber, & 
Torney-Purta, 2006). In particular, civic knowledge as a critical student learning outcome grows 
in importance with growing diversification of societies, increased sensitivity to individual and 
group identities, and concern about effectiveness of preparing young people for future adult roles 
as creative and thinking citizens (Pigozzi, 2006). 

Another central issue is whether the results of international assessments are actual and appro- 
priate measures of student achievement. The media and policymakers seem to believe so. For 
example, the Wall Street Journal reported that U.S. teenage students are among the worst at math 
because the United States ranked 24th among 29 OECD countries in mathematics literacy in 
PISA 2003 (Kronholz, 2004). U.S. Education Secretary Arne Duncan likened mediocre scores of 
American students on the PISA 2009 exam to “an absolute wake-up call for America” (“In Rank- 
ing,” 2010). However, one must ask, Did the Wall Street Journal and Arne Duncan reasonably 
and legitimately interpret the PISA results, and are the international assessment results clear-cut 
evidence of learning achievement? If the average student achievement scores of a country in 
reading, scientific, or mathematical literacy are considerably higher than those in another coun- 
try, it might be automatically inferred that the education system of a high-performing country 
is more effective and advanced than a low-performing country. However, this article assumes 
that it is somewhat hasty to take the international assessment results at face value. Instead, the 
international assessment results need to be interpreted and reassessed after taking other relevant 
factors into account, which is a way to gain more insight into the quality of education. This issue 
is of particular importance due to the rising costs of education in conjunction with persistent 
concerns about quality that have called efficiency in education into question (Levin, Jamison, & 
Rander, 1976). 

Given these two issues, the purpose of this article is twofold. First, this article explores both 
cognitive achievement and civic knowledge achievement as the central metric of educational 
outcomes by comparing them to each other and analyzing how they correlate. As an outcome of 
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citizenship education, civic knowledge is regarded as a key cognitive component of citizenship 
that leads both to skills in understanding political communication and to attitude toward partici- 
pation in conventional citizenship behavior (Torney-Purta, 2002; Torney-Purta & Lopez, 2006). 
This analysis is expected to suggest that cognitive achievement is just one of the educational out- 
comes. Second, this article intends to reinterpret the international assessments results by utilizing 
efficiency analysis (also called production-function approach) concerned with the production of 
outputs and with inputs, as education is a production process combining inputs to gain desired 
outputs (Mun, 2002). Efficiency is counted as one of the quality indices that are accessible to 
a variety of stakeholders in the educational process (Tattersall, Waterink, Hoppener, & Koper, 
2006), as it can be understood as the extent to which educational systems are successful in opti- 
mizing the relationship between educational inputs and outputs (UNESCO, 1995). Consequently, 
this article provides some insights into reading and interpreting international student assessments 
adequately. 

The remainder of this article is organized as follows: The first section clarifies data and 
indicators for efficiency analysis. The second section discusses the comparison of cognitive 
achievement and civic knowledge achievement, and the third section presents the results of 
efficiency analysis. The fourth section provides a brief discussion of the findings. Finally, this 
article ends with a conclusion. 


DATA AND INDICATORS 


This article uses the PISA! results for cognitive achievement? and the IEA Civic Education Study 
(CIVED)? results for civic knowledge achievement. To compare cognitive achievement with 
civic knowledge achievement, PISA 2000 and CIVED 1999% are selected. These two assessments 
intend to measure the achievement of comparable and representative samples of students in 
different subjects and in different school systems.* On the other hand, this article uses PISA 
2009° results to measure educational efficiency only due to truly limited availability of data on 
inputs for civic education. 


'The PISA evaluates not the extent to which students master a specific school curriculum but the ability to use the 
wider knowledge, concepts, and skills in core subjects to meet real-life situation (OECD, 2001). 

*Cognitive achievement is interchangeably used with student learning achievement on three core subjects (mathemat- 
ics, reading, and science) in this article. 

3CIVED basically assesses the content of students’ civic knowledge and their civic attitudes and engagement. 
Specifically CIVED consists of five components: (a) civic knowledge; (b) skills in interpreting civic information; (c) 
concepts of democracy, citizenship, and government; (d) attitudes toward the nation, the government, immigrants, and 
women’s political rights; and (e) willingness to participate in civic-related activities (Torney-Purta, Lehmann, Oswald, & 
Schulz, 2001). 

4CIVED 1999 was conducted in two phases. This article focuses on the second phase only in which nationally 
representative samples were tested on their civic knowledge and skills. 

Sample students who participated in both assessments were not identical to each other. However, considering that 
CIVED was carried out | year before PISA was conducted, target populations appear to be the same. Target populations are 
based on the coverage of a specific age cohort: in PISA 15-year-olds and in CIVED 14-year-olds enrolled in educational 
institutions. Especially because this article compares national averages on international assessments, comparison of PISA 
2000 and CIVED 1999 is reasonable. 

PISA 2009 results in lieu of PISA 2000 are used mainly because of data availability. 
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Educational efficiency requires agreement on measurement methods (Reimann, 2004) because 
it is an indicator of the quality of educational processes. This article analytically depends on 
the standard definition of efficiency—the ratio of output to input (Rumble, 1997)—which is 
specifically defined as internal efficiency (Mun, 2002). Internal efficiency in education indicates 
a comparison of educational outputs to the costs of educational inputs (European Research 
Associates, 2006). In this article, educational output is student test scores on PISA, and two 
educational inputs are examined: learning time and education spending. Student learning time 
is the most valuable resource in the educational process (Haahr, Nielsen, Hansen, & Jakobsen, 
2005), and education funding is one of the most significant issues for debates on improving student 
achievement, though empirical evidence on the impacts of education funding are mixed (Fleisch, 
2004; Lopus, 1990; Ram, 2004). Therefore, efficiency is estimated as the ratio of mean scores in 
PISA 2009 (the numerator) to the amount of time in hours and of money spent (the denominator) 
with the goal of exploring the relationships between differences in student achievement and 
differences in two inputs. 

Furthermore, this article focuses on two different dimensions of efficiency: technical efficiency 
(TE) and allocative efficiency (AE). TE refers to maximizing feasible outputs from a given set 
of inputs, and AE refers to the ability to produce at a given level of output, minimizing the costs 
of input (Levin et al., 1976). Both, however, will identify the same set of efficient producers 
(Afonso, Schuknecht, & Tanzi, 2006). Educational efficiency is thus examined in terms of (a) 
test scores that students achieve for $1 spent and for | hr devoted to learning (TE) and (b) the 
amount of educational expenditure and learning time to increase one PISA scale score point (AE). 
Consequently, greater educational efficiency is generally indicative of the ability of students to 
achieve high scores with the least amount of time and/or money (Vlasceanu, Grunberg, & Parlea, 
2007). This article also presents efficiency in a cross-national comparative sense. Namely, the 
relationship between outputs and inputs in country A are compared with that of another country. 

Using these data, this article conducts a cross-national comparative study exploring and 
comparing the outcomes from the education systems of 17 countries’ that participated in both 
PISA and CIVED. Examination of efficiency also concentrates on these 17 countries only in 
order to draw overarching conclusions. Results of cross-national comparison are reported in a 
descriptive way. In particular, graphical methods of data summarization are employed to present 
more understandable summaries of educational efficiency analyses and report differences and/or 
similarities between efficiency levels and country rankings. 


COGNITIVE ACHIEVEMENT VERSUS CIVIC KNOWLEDGE 
ACHIEVEMENT 


There does not appear to be a strong correlation between civic knowledge achievement and 
cognitive achievement on three core subjects, whereas more clear-cut linear relationships between 
one core subject and another are observed (see Figure 1). This visual inspection aligns with 
the values of correlation coefficients. The correlations of one core subject with another are 
0.81 (reading and math), 0.82 (math and science), and 0.92 (reading and science), whereas the 


7The 17 participating countries are Australia, Belgium, Czech Republic, Denmark, Finland, Germany, Greece, 
Hungary, Italy, Latvia, Norway, Poland, Portugal, Russian Federation, Sweden, Switzerland, and the United States. 
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FIGURE 1 Relationship of one subject to another in PISA 2000 and ICVED 1999. Source: OECD (2001, 2010, 2011) 
and Schulz and Sibberns (2004) (color figure available online). 


correlations of civic knowledge with core subjects are 0.30 (science), 0.32 (reading), and —0.05 
(math). These findings imply that each country places different levels of emphasis on civic 
knowledge and knowledge of the core subjects. 

This implication is reflected in civic knowledge rankings in Table | that shows differ- 
ences/changes in rankings in the core subjects and civic knowledge. A simple comparison of 
these rankings may lead to a misleading interpretation. Yet there are two pronounced points. 
First, rankings of four countries (the United States, Italy, Poland, and Greece) in civic knowIl- 
edge are higher than those in cognitive achievement. Second, the opposite case is observed in 
some countries, in particular Sweden, Switzerland, and Belgium, whose rankings in civic knowl- 
edge achievement turn out to be lower than those in cognitive achievement. In terms of the 
quartile distributions of rankings among all the participating countries, the former group belongs 
to the bottom quartile or lower middle quartile in PISA rankings but to the top quartile in CIVED 
rankings. Conversely, the latter group falls from the upper middle quartile in PISA ranking to the 
bottom or the lower quartile in CIVED ranking. In addition, Eastern European countries (Russia, 
Czech Republic, Poland, and Hungary), with the exemption of Latvia,® and countries with his- 
torically centralized authoritarian rule such as Germany and Italy, generally seem to regard civic 
knowledge achievement as important, considering their higher rankings in CIVED. Rankings of 
the other five countries (Finland, Australia, Norway, Denmark, and Portugal) are relatively static. 


‘Among 17 sample countries, Latvia is at the bottom of rankings in both core subjects and civic knowledge. 
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TABLE 1 
Student Achievement in PISA 2000 and CIVED 1999 and Rankings 
Reading Mathematical Scientific Sum of PISA Civic 

Country Literacy Literacy Literacy Test Scores Knowledge 
ee eee 
Finland 546 (1) 536 (4) 538 (3) 1620 (3) 109.3 (2) 
Australia 528 (4) 533x(5)) 528 (7) 1589 (6) 101.7(11) 
Sweden 516 (9) 516 (15) 512 (10) 1544 (10) 99.1 (18) 
Belgium 507 (10) 520 (9) 496 (17) 1523 (11) 94.7 (22) 
Norway 505 (13) 499 (17) 500 (13) 1504 (15) 102.9 (9) 
United States 504 (15). 493 (19) 499 (14) 1496 (17) 106.5 (6) 
Denmark 497 (16) 514 (12) 481 (22) 1492 (18) 100.4 (14) 
Switzerland 494 (17) 529 (14) 496 (17) 1519 (13) 98.3 (19) 
Czech Republic 492 (19) 498 (18) 511 (11) 1501 (16) 102.6 (10) 
Italy 487 (20) 457 (26) 478 (23) 1422 (24) 105.4 (7) 
Germany 474 (21) 490 (20) 487 (20) 1451 (21) 99.8 (15) 
Hungary 480 (23) 488 (21) 296 (15) 1464 (20) 101.6 (12) 
Poland 479 (24) 470 (24) 483 (21) 1432 (23) 110.6 (1) 
Greece 474 (25) 447 (28) 461 (25) 1382 (27) 107.9 (4) 
Portugal 470 (26) 470 (24) 459 (28) 1399 (26) 96.2 (21) 
Russian Federation 462 (27) 478 (22) 460 (26) 1400 (25) 99.6 (16) 
Latvia 458 (28) 462 (25) 460 (27) 1380 (28) 91.5 (26) 
OECD average 500 500 500 1500 100 





Note. Source: OECD (2001) and Schulz and Sibberns (2004). Numbers in parentheses are rankings among all 
participating countries in the Programme for International Student Assessment (PISA) and the IEA Civic Education 
Study (CIVED), respectively. Average of civic knowledge is international average, not OECD. 


Of interest, among 17 sample countries, Finland ranks first in all the four components of educa- 
tional outcomes, which implies that it pursues both cognitive achievement and civic knowledge 
achievement in a desirably balanced way. 

Figure 2 ° illustrates mean scores for all the 17 sample countries in both PISA 2000 and CIVED 
1999. Figure 2 presents an especially interesting point—the degree of balance between cognitive 
achievement and civic knowledge achievement. A dashed diagonal line indicates a perfect balance 
between civic knowledge achievement and cognitive achievement. Finland appears to be the top 
performing country as regards to the balance between civic knowledge achievement and cognitive 
achievement. This finding may become another source of pride for the Finnish education system. 
Similarly to Finland, Denmark and Latvia are close to a perfect balance. However, considering 
that the extent of desirability of the balance increases as we go along the diagonal line from 
the bottom left-hand to the top right-hand, Finland is much more desirably well balanced than 
Denmark and Latvia. In particular, Finland is in the top quartile of student achievement in both 
PISA and CIVED, but Denmark is in the middle quartile and Latvia in the bottom quartile. In 
addition, this article categorizes countries below and to the right of the diagonal as cognitive 


*Student test scores are converted to a scale of 0 to 10 by the author. Initial range of sum of test is 1400 to 1600 for 
core subjects and 90 to 110 for civic knowledge. Reference points in converting initial test scores are OECD average, 
1500, for core subjects and international average, 100, for civic knowledge, which means that 5 on a scale of 0-10 is 
international average. Scores below 1400 in core subjects and 90 in civic knowledge are converted to 0, scores above 


1600 and 110 to 10. 
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FIGURE 2 Dispersion of student achievement in PISA 2000 and CIVED 1999. Source: OECD (2001, 2010, 2011) 
and Schulz and Sibberns (2004). Nere. AUS = Australia; BEL = Belgium; CZE = Crech Republic; DNK = Denmark; 
FIN = Finland; GER = Germany; GRE = Greece; HUN = Hungary; ITL = Italy; LAT = Latvia; NOR = Norway; POL 
= Poland; POR = Portugal: Russia = Russian Federation; SWE = Sweden; SWI = Switzerland; USA = United States 
(color figure available online). 


achievement-skewed (CAS) and those above and to the left of the diagonal as civic knowledge- 
skewed (CKS), respectively. Australia, Belgium, Switzerland, and Sweden fall under the group 
of CAS and the rest of countries except for Finland and Denmark, under CKS.!° Countries with 
lower rankings in civic knowledge achievement in Table | can be generally labeled as CAS 
countries and those with lower rankings in cognitive achievement as CKS countries. 

Figure 2 additionally displays four cells (I, II, II], and IV) divided by two solid lines that are 
international averages. Cell I indicates below-international-average achievement in core subjects 
and civic knowledge, Cell II above-international-average achievement in civic knowledge only, 
Cell II] above-international-average achievement in both, and Cell IV above-international-average 
achievement in core subjects only. Latvia, Portugal, Russian Federation, and Germany under Cell 
I can be categorized into low-achieving countries in both core subjects and civic knowledge. 
Finland, Australia, and Norway under Cell III may be regarded as relatively well-balanced and 
high-achieving countries in not only civic knowledge achievement but also cognitive achievement. 
Sweden, Switzerland, and Belgium are high-achieving countries in cognitive achievement only, 
whereas Poland, Greece, Italy, Hungary, Denmark, and the United States are high-achieving 
countries in civic knowledge only. However, it should be kept in mind that there are some 
within-category variations in student achievement in addition to between-category variations. 


‘Although not shown in this article, the scatter plots of mean scores in each of core subjects and civic knowledge are 
truly similar as the scatter plot shown in Figure 2. 
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FIGURE 3 Relationship between student achievement in PISA 2009 and cumulative spending. Source: OECD (2010, 
2011). Note. AUS = Australia; BEL = Belgium; CZE = Crech Republic; DNK = Denmark; FIN = Finland; GER 
= Germany; GRE = Greece; HUN = Hungary; ITL = Italy, LAT = Latvia; NOR = Norway; POL = Poland; POR 


= Portugal; Russia = Russian Federation, SWE = Sweden; SWI = Switzerland; USA = United States (color figure 
available online). 


For instance, among countries under Cell II, Australia is more of a high achiever in PISA, 
Norway in CIVED, and Finland in both, respectively. Consequently, when educational outcomes 
are assessed and compared cross-nationally on a basis of both cognitive achievement and civic 
knowledge achievement, the relative rankings of countries change. Countries falling under Cell II, 
in particular Finland, may be considered benchmarking countries in student learning achievement. 


EFFICIENCY ANALYSIS 


Education Spending Per Student 


Cumulative education expenditure by educational institutions per student is used as a proxy for the 
measure of education spending per student as in Haahr et al. (2005). This is reasonable now that the 
educational process is cumulative (Hanushek, 1987) and that student achievement is the outcome 
of a long-term and cumulative process of knowledge attainment (Baumert, Ludtke, Trautwein, & 
Brunner, 2009). First, a moderate linear relationship is identified between student achievement 
and cumulative spending!' (see Figure 3) and there is a positive relationship between the wealth of 
a country and cumulative spending per student (see Table 2). These relationships do not indicate 


'lThe correlation between them is 0.48 with its p value of .0S. 


‘ 
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TABLE 2 


Student Performance in PISA 2009 and Cumulative Education Spending per Student 
Se ee ee eee 


Country Sum of Test Scores Score Ranking Spending (US$) TE Ratio Ranking AE 

(DEES BE a a es ee ee eee ee ea 
Finland 1,631 | 71,385 0.023 7 43.77 
Australia 1,589 2 72,386 0.022 8 45.55 
Switzerland ayy 3 104,352 0.015 14 67.23 
Belgium 1,528 4 80,145 0.019 10 $2.45 
Poland 1,503 a 39,964 0.037 2 26.59 
Norway 1,501 5 101,265 0.015 14 67.47 
Denmark 1,497 7 87,642 0.017 12 58.55 
United States 1,496 8 105,752 0.014 16 70.69 
Sweden 1,486 9 82,753 0.017 12 55.69 
Czech Republic 1,471 10 44,761 0.033 3 30.42 
Portugal 1,469 1] 56,803 0.026 6 38.67 
Hungary 1,464 2 44,342 0.033 3 30.29 
Germany 1,461 13 63,296 0.023 7 43.32 
Latvia 1,460 14 — — — — 

Italy 1,458 15 77,310 0.019 10 53.02 
Greece 1,419 16 48,422 0.029 5 34.12 
Russian Federation 1,405 li 17,499 0.080 1 12.45 
OECD average 1,500 69,135 0.021 46.09 





Note. Source: OECD (2010, 2011). Sum of test scores is just the sum of three core subjects, reading, mathematical 
and scientific literacy. Rankings are based on sample countries this article examines only. Cumulative education spending 
is in equivalent U.S. dollars converted using PPPs. Technical efficiency (TE) is ratio of scores to expenditure, that is, 
test scores achieved when $1 is spent, and allocative efficiency (AE) is an average expenditure to get | test score point. 
Both of them are calculated by the author. PISA = Programme for International Student Assessment. 


that wealthy countries and more spending are more efficient in improving student achievement 
scores, however. The opposite seems to be a reasonable interpretation. According to Table 2,!? 
the top three high-spending countries—the United States, Switzerland, and Norway—rather rank 
in the bottom quartile of ratio of scores to spending (see column 5), that is, their TEs are among 
the lowest. U.S. students achieve 0.014 PISA scale score point for every $1 in spending, and 
Norwegian students and Swiss students 0.015, respectively. These countries’ efficiency level is 
lower than OECD average. On the contrary, Russian Federation, which achieves 0.081 point for 
every $1 in spending, ranks number one in TE, followed by Poland, Hungary, and Czech Republic. 
These interpretations are true in AE as well (see column 7). Russian Federation spends an average 
of $12.45, which is just 17% of the highest spending country, the United States ($70.69), to gain 
I score point, though its sum of test scores is the second lowest. Other efficient countries are 
Poland, Hungary, Czech Republic, and Greece, most of which, however, are not categorized as 
high-achieving countries in sum of test scores (see column 2 in Table 2). Top scoring Finland 
spends $43.77, which is about average, but accounts for 62% of the United States, and achieves 
0.023 score point for every $1 in spending. Consequently, the relationship between cumulative 


!2Qutcomes in PISA 2003 are similar to those specified in Table 2, which is not shown in this article. The author just 
checked whether any big changes are observed. Poland is the most cost-effective country in PISA 2003 for which Russian 
Federation is unavailable. 


A CROSS-NATIONAL COMPARATIVE STUDY 115 


spending and student achievement at least is far from deterministic, which indicates that there is 
between-country variation in educational efficiency as specified in Table 2. 


Time (in Hours) Devoted to Learning 


In contrast to education spending per student, learning hours devoted to core subjects are nega- 
tively related to student achievement (see Figure 4). The correlation between them is —0.54 with 
its p value of .02. This negative relationship is somewhat extreme, considering findings from 
previous studies positing a weak statistical relationship between time spent on learning and stu- 
dents’ average achievement scores on PISA due to other factors such as differences in students’ 
ability to learn and the amount of homework given to students by their teachers (e.g., Fuchs & 
Woessmann, 2004; Haahr et al., 2005). Regardless of whether the relationship between them is 
negative or weak, it implies that the net effect of time devoted to learning is varied both within 
and across countries, that is, some students in some countries spend, either efficiently or ineffi- 
ciently, their time on studying. Greece, Russian Federation, Italy, and Latvia appear to have low 
efficiency, whereas Finland, Switzerland, Sweden, and Australia have relatively high efficiency 
(see Figure 4). Table 3 specifically reports between-country variation in educational efficiency in 
learning time. Overall, efficiency rankings in learning hours differ to some extent from efficiency 
rankings in education spending. The most pronounced difference is that countries with a high 
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FIGURE 4 Relationship between student achievement in PISA 2009 and total learning hours. Source: OECD (2010, 
2011). Note. AUS = Australia; BEL = Belgium; CZE = Crech Republic; DNK = Denmark; FIN = Finland; GER 
= Germany; GRE = Greece; HUN = Hungary; ITL = Italy, LAT = Latvia, NOR = Norway; POL = Poland; POR 
= Portugal; Russia = Russian Federation; SWE = Sweden; SWI = Switzerland; USA = United States (color figure 
available online). 
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TABLE 3 
Student Performance in PISA 2009 and Time Devoted to Learning 


eee 


Country Sum of Test Scores Score Ranking Total Hours TE Ratio Ranking AE 
Pa ed on we ne ee 2 Bs es ee ee ee ee eee 
Finland 1,631 I 14.16 3.03 1 0.33 
Australia 1,589 2 17.85 223 ff 0.45 
Switzerland ee 3 15.66 AS 3 0.36 
Belgium 1,528 4 15.43 2.69 4 0.37 
Poland 1,503 5 20.19 2.02 11 0.49 
Norway 1,501 5 16.79 2:35 5 0.43 
Denmark 1,497 ii 21.94 1.63 17 0.61 
United States 1,496 8 19.69 1.96 13 0.51 
Sweden 1,486 9 14.37 29 2 0.36 
Czech Republic 1,471 10 15.98 2.30 6 0.44 
Portugal 1,469 11 17.94 1.95 14 0.51 
Hungary 1,464 12 17.89 2.21 8 0.45 
Germany 1,461 13 18.44 1.98 12 0.50 
Latvia 1,460 14 20.03 2.08 10 0.48 
Italy 1,458 15 21.07 1.82 i) 0.55 
Greece 1,419 16 21.48 1.65 16 0.61 
Russian Federation 1,405 17 19.13 2.16 9 0.46 
OECD average 1,500 17.40 2.20 0.45 


Note. Source: OECD (2010, 2011). Rankings are based on sample countries this paper examines only. Total learning 
hours include regular school lessons (direct instruction time), out-of-school time lessons, and individual study and are 
mean learning hours per week. In calculating ratio of scores to time, time is simple multiplication of total learning hours 
peer week and instructional weeks in years (school days) and time for | point indicates an average of learning hour for 
one test score point. Both of them are calculated by the author. Technical efficiency (TE) is ratio of scores to learning 
time and allocative efficiency (AE) learning time spent to earn one test score. PISA = Programme for International 
Student Assessment. 


efficiency level in expenditures such as Greece, Poland, and Portugal show up as having a low 
efficiency level in learning hours, whereas the opposite trend is observed in such countries as 
Switzerland, Norway, and Sweden. Highly efficient countries in educational spending, Russian 
Federation and Hungary, retain approximately an average efficiency in learning time.!* Australia 
and Czech Republic still have an average efficiency level. The highest achieving country, Finland, 
shows up as the most efficient. Finnish students achieve 3.03 PISA scale score points for every 
hour of study and spend 33 min to gain one test score point (see columns 5 and 7). Conversely, 
Demark is labeled as the least efficient country, followed by Greece, Italy, and Portugal (see 
column 6). Danish students gain 1.63 score points for every hour of study and spend 61 min to 
earn | test score point, which is 54% of the time Finnish students devote. Accordingly, it may be 
said that Finnish students are about twice as efficient as Danish students to gain 1 score point. 
Overall, efficiency levels of countries are generally not similar when measured using education 
expenditures and learning time as educational inputs. 


l3The average TE level is 2.21 and the average AE level is 0.46. 
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The Combination of Two Inputs 


The “least cost” set of input producing any given output at minimum cost (Hanushek, 1987) 
is another component of educational efficiency. Following the concept of the least-cost set of 
input, Figures 5a and Figures 5b depict the intersection of cumulative expenditure and time 
spent on learning. Figure Sa illustrates TE of each country, and Figure 5b illustrates AE. As in 
Figure 2, in Figures 5a and Figures 5b, there are four cells divided by OECD average. Cell I 
indicates below-OECD-average efficiency in education expenditure and learning time, Cell II 
above-OECD-average efficiency in education expenditure only, Cell III above-OECD-average 
efficiency in both, and Cel! IV above-OECD-average efficiency in learning time only. Finland, 
Hungary, Czech Republic, and Australia under Cell III] can be classified as having high TE in 
both inputs, whereas Italy, the United States, and Demark under Category I as having low TE. 
Division of categories in Figure 5b is opposite to those in Figure 5a, that is, Cell I indicates above- 
OECD-average efficiency in both inputs, Cell II below-OECD-average efficiency in both, Cell 
Il above-OECD-average efficient in learning hours only, and Cell IV in educational spending 
only. Finland and Czech Republic under Cell I have high AE in both inputs, but the United 
States, Italy, and Demark under Cell III have low AE. Australia is about OECD average in 
both inputs. 
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FIGURE 5a __ Dispersion of technical efficiency of each country. Source: OECD (2010, 2011). Note. AUS = Australia; 
BEL = Belgium; CZE = Crech Republic, DNK = Denmark; FIN = Finland; GER = Germany; GRE = Greece, HUN = 
Hungary; ITL = Italy; LAT = Latvia; NOR = Norway; POL = Poland; POR = Portugal; Russia = Russian Federation; 
SWE = Sweden; SWI = Switzerland; USA = United States (color figure available online). 
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FIGURE 5b __ Dispersion of allocative efficiency of each country. Source: OECD (2010, 2011). Note. AUS = Australia; 
BEL = Belgium; CZE = Crech Republic; DNK = Denmark; FIN = Finland; GER = Germany; GRE = Greece; HUN = 
Hungary; ITL = Italy; LAT = Latvia; NOR = Norway; POL = Poland; POR = Portugal; Russia = Russian Federation; 
SWE = Sweden; SWI = Switzerland; USA = United States (color figure available online). 


DISCUSSION 


Comparison of civic knowledge achievement and cognitive achievement (Table 1 and Figure 2) 
intends not to provide evidence to prove any theory but to show the average level of student 
achievement in different subjects in each country. It is clear that the average student achievement 
level of and rankings of participating countries in PISA are not generally similar to those in 
CIVED with the exception of Finland and Latvia (see Table 1). For instance, Greece and Poland 
appear to be exceptional countries in that their achievement and rankings are in the top quartile in 
civic knowledge achievement, but they are in the bottom quartile in cognitive achievement. Yet, 
there are two exceptions: Finland is at the top in PISA and near the top in CIVED, whereas Latvia 
is at the bottom in both. Therefore, caution is necessary when discussing student achievement 
from a cross-country comparative perspective. 

The description of this study poses questions for further study. High-performing countries in 
PISA such as Australia, Switzerland, and Belgium appear to be CAS. Is it then reasonable to 
accept that the perspective of previous studies, for example, negligence of civic education due 
to growing pressure to teach curricular areas included in compulsory assessments (Hinde, 2008), 
loss of civic education as a subject holding a rock-solid place in an international core curriculum 
(Schulz & Sibberns, 2004), are true in these CAS countries? Conversely, about two thirds of the 
sample countries are CKS. Then, is it legitimate to assume that these CKS countries consider 
civic knowledge achievement as a component of education equally important to core subjects? 


A CROSS-NATIONAL COMPARATIVE STUDY 119 


There are other intriguing topics for further study. Finland shows up as the best country, 
considering its high efficiency and its students’ high performance in both PISA and CIVED, 
followed by Australia and Czech Republic. Czech Republic is an interesting country, given its 
mediocre student achievement (even lower than OECD average) in PISA 2009, but its relatively 
high efficiency. On the contrary, the United States and Denmark show up as worse performing, 
because their rankings in efficiency are even lower than their rankings in PISA. Meanwhile, as 
previously stated, there are two conflicting features. Such countries as Sweden, Switzerland, and 
Norway have high efficiency in learning time but low efficiency in education spending. Greece, 
Portugal, Russian Federation, and Poland are the opposite. Then, can it be assumed that the former 
group provides students with high-quality and expensive education intensively, whereas the latter 
group provides poor-quality and inexpensive education for a long period? There is another two- 
comparable-group: Finland and Australia (high efficiency in time and money) and the United 
States and Denmark (low efficiency in both). In this context, it is legitimate to investigate what 
makes these countries different, that is, what leads to either high- or poor-student achievement. In 
particular, it would be interesting to explore what makes Finland ranked first in PISA and second 
in CIVED. 

On the other hand, the findings that this article presents are not all-inclusive and deterministic 
for two reasons. First, exclusively concentrating on the quantitative amount of education expen- 
diture and learning time may lead to an inaccurate measure of efficiency levels. According to the 
PISA (2010) report, cumulative expenditure can take the form of buildings and infrastructure, 
salaries paid to teachers, administrators and support staff, and transportation and meals for stu- 
dents. However, because the method of expenditure distribution is not clarified, the qualitative 
aspect of expenditure is difficult to recognize. How to use money is as important as absolute 
quantitative amounts of spending. For instance, all the 17 countries prioritize small class size 
and low teachers’ salaries in allocating resources for education, but as previously discussed, 
efficiency levels are different. Conversely, such countries as Japan and Korea allocating resources 
with the priority of large class size and high teachers’ salaries are categorized into high-achieving 
countries in PISA. In addition, not only the quantitative amount of learning time but also the 
qualitative aspect of learning time (i.e., whether time is effectively devoted to adequate con- 
tents of learning) is a potential factor affecting student achievement. It is obvious that learning 
productivity varies by student, because students’ socioeconomic background may influence, for 
example, their homework practices, with the more advantaged students who are highly likely to 
have better opportunities to work efficiently (Haahr et al., 2005). In this regard, the educational 
efficiency just estimated is not firmly deterministic, leaving substantial scope for improving the 
efficiency of educational systems. 

Second, more inclusive and detailed studies are required due to the fact that educational outputs 
can result from a variety of different combinations of inputs besides educational spending and 
learning time. Different combinations of inputs may be explained in two ways: extension of inputs 
examined in this article and inclusion of new types of inputs. When examining the efficiency of 
education expenditures, this article has focused on cumulative expenditure by institutions only, 
due to comparable data availability. But, given the need of microlevel analysis of educational 
decisions that individuals and/or households make, the efficiency of education expenditure needs 
to consider expenditures that the private sector spends in the education production process, 
in particular for countries where out-of-school learning is somewhat ordinary. Collecting, in 
particular cross-nationally comparable, household-level data are a big challenge. Other new 
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exemplary inputs that previous studies generally examine are teacher characteristics, class size, 
family background, and curriculum (e.g., Jepsen, 2005; Nye, Konstantopoulos, & Hedges, 2004; 
Rivkin, Hanushek, & Kain, 2005; Woessmann, 2004). It is thus necessary to consider if other 
inputs are adequately integrated into the measurement of educational efficiency. Integrating other 
inputs into efficiency analysis will provide more insight into the interpretation of international 
student assessment results. 


CONCLUSIONS 


Attracting significant attention from the media and policymakers, international student assess- 
ments are serving as pressures on and challenges for individual countries. To illustrate, the number 
of press coverage pages in the PISA participating countries about 1 month after the public release 
of the results is 687 in Germany, 149 in Switzerland, 54 in Australia, and 36 in the U.S. (Lin- 
nakyla, 2006). Even so, paying full attention only to publicly released rankings in international 
student assessments may lead to superficial interpretations of the assessment results. Simply 
equating the rankings with the quality of the educational system (i.e., educational outcomes) can 
be a questionable perspective because the assessment results can be interpreted differently. In 
this context, one pertinent issue is whether we have documented conclusive evidence that the 
assessment results are true measures of student learning achievement of participating countries. 
In support of this position, this article has explored student achievement in the PISA tests in 
comparison with civic knowledge achievement in CIVED across 17 countries. This article has 
also evaluated efficiency in educational processes by assessing outputs (student performance in 
PISA) against inputs (educational spending and learning time) directly used in the education 
system. These two approaches clarify that publicly released rankings in PISA turn out to change 
to some extent when the rankings are reevaluated by taking other factors into consideration. This 
implication should be considered when interpreting the results of international assessment and 
the relative rankings of participating countries. 

These two approaches are of importance to avoid a hasty conclusion about a country’s education 
system. International student assessment results in some cases might provide “bragging rights,” 
in particular for high-performing countries. Simultaneously, for some poor-performing countries, 
unsatisfactory results might trigger reform-minded policymakers to execute policy changes to 
catch up with high-performing countries and to improve learning achievement of their own 
students. However, as previously argued, the use of international assessment results should be 
treated and understood with great caution. Cross-national comparisons on a basis of international 
student assessments are beneficial to a great extent, in particular by encouraging improvements 
in educational investments, the establishment of new standards for learning achievement and 
curriculum development, and raising awareness of importance of school resources and teaching 
practices (Beaton, Postlethwaite, Ross, Spearritt, & Wolf, 1999; Postlethwaite, 2004). Even so, it 
is not recommended that national average scores per se become the sole foundation for education 
policy decisions on student achievement because of profound variation in educational systems 
between countries. It is crucial to have a sufficiently broad and long-term perspective rather 
than a narrow view of educational outcomes. Other relevant factors should be allowed for in 
interpreting cross-national comparisons. Learning time and educational expenditures are just 
exemplary factors. 
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With globalization, the world has become more interconnected and interdependent, with people, 
capital, and ideas rapidly migrating across borders. Yet, along with greater global interdependence 
and increased diversity within societies, economic and social inequalities have deepened, making 
poverty one of the leading global problems. To lessen poverty around the world, better integrate 
immigrant groups into societies, and enhance stability, both societies and education systems are 
dedicated to fostering social cohesion as a key objective. Yet, still little is known about the extent to 
which immigration and socioeconomic status are predictive of social cohesion. Using cross-national 
data from the 2009 International Civic and Citizenship Study for 38 countries, this article further 
investigates the link between poverty, immigration, education, and social cohesion. Findings indicate 
that socioeconomic status is an important influence on social cohesion of immigrant students and that 
schools play an important role in fostering social cohesion. 


The world has become more interconnected and interdependent, with people, capital, and ideas 
rapidly migrating across borders (McGrew & Held, 2002). Yet, along with greater global interde- 
pendence, economic and social inequality and marginalization have deepened (e.g. Sen, 2001). 
For example, economic inequalities have worsened in the wake of the recent global financial 
crisis, as global growth has stalled, making poverty one of the leading global problems (World 
Bank, 2011). Moreover, the movement of people has brought increased diversity within societies, 
raising concerns that societies are becoming more divided along cultural and ethnic lines (Banks, 
2007). Together, both the “shifting centre of economic gravity” (OECD, 2011, p. 15) and the 
increase in diversity present education systems around the world with a range of challenging 
responsibilities: from preparing students with 21st-century skills needed for the global market- 
place to instilling common values across diversifying national societies. 

To better integrate immigrant groups into societies, and create more stable societies, social 
cohesion—the metaphorical glue that holds a society together—has been considered by some 
as one of the key components that societies and education systems must emphasize (Silova, 
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Johnson, & Heyneman, 2007). Increasingly, international organizations and national governments 
are highlighting the importance of social cohesion to global growth and development, peaceful 
and democratic societies, and greater equity, citing education as the key space for governments to 
foster the ideas that contribute to a socially cohesive society and alleviate poverty (Hulse & Stone, 
2007; OECD, 2011). Yet, the diverse populations that are often the targets of social cohesion efforts 
greatly differ and include large variations in socioeconomic status (SES). To that end, drawing 
on cross-national data from 38 countries, this article investigates the link between immigration 
and social cohesion and the ways in which differences in SES moderate that relationship. By 
exploring cross-nationally whether immigration and SES are predictive of social cohesion, the 
article contributes to conversations about the role that schools have in fostering social cohesion 
and better integration of immigrant students in schools (Flores & Loss, 2010; Heyneman, 2010; 
Janmaat & Mons, 2011; Sudrez-Orozco, 2001; Sudrez-Orozco, Sudrez-Orozco, & Sattin-Bajaj, 
2010). 


GLOBALIZATION, IMMIGRATION, AND INEQUALITY 


There are two faces to the 21 st-century global, mobile, and interconnected world: One that presents 
a sense of optimism about the potential for increased cross-cultural communication, democratic 
forms of governance, and peace-building, and the other that is marked by “increased violence, 
exclusion, and growing inequality” (Appadurai, 2006, pp. ix—x). To frame our study, we draw on 
Bauman’s (1996) early work on globalization, in which he discussed global migration and made 
a distinction between two groups: tourists and vagabonds. Tourists, he argued, are individuals 
who freely move across borders and whose economic means, as well as their cultural capital 
(Bourdieu, 1986) make them more welcome migrants. Vagabonds, on the other hand, are those 
who are globally mobile not by choice, but often are forced into migrating for economic, political, 
social, or environmental reasons. As poor immigrants, vagabonds are not typically welcome, and 
are often the root of heightened social anxieties over shifting demographics (Appadurai, 2006). 
In this article, we contend that national societies are challenged by increased immigration, both 
of tourists and vagabonds. As nation-states increasingly look to social cohesion as an avenue to 
reduce marginalization and social fragmentation, accounting for differences and views among 
both the tourists and vagabonds may be crucial for the development of interventions and policies 
that will increase cohesion. 


Immigration and Inequality 


The devastating impact of poverty and inequality is far reaching, with consequences that resonate 
through national economies and societies. Recent research has shown that rising income inequality 
within countries is related to a number of important outcomes, including a decline in well-being, 
trust, public health, and overall social cohesion in societies (Wilkinson & Pickett, 2010). Likewise, 
other research illustrates the link between higher levels of income equality and trust among citizens 
in the social and political systems of society (Green, Preston, & Janmaat, 2006). These studies 
point to the alleviation of income inequality as an important aspect of increasing socially cohesive 
societies. 
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In addition, with globalization, national societies have arguably become more ethnically and 
culturally diverse, resulting in very few countries that are comprised of largely homogeneous 
populations (Inglis, 2008; Suarez-Orozco, 2001). It was estimated that in 2013 there were around 
232 million international migrants worldwide (United Nations, 2013), with the trend in foreign- 
born residents increasing in many countries. In the United States, for example, one in eight U.S. 
residents is foreign-born (U.S. Bureau of Census, 2010). In addition, across the European Union 
(EU), there were 48.9 million foreign-born residents in 2011 (9.4% of the population), of which 
32.4 million were born outside of the EU (Eurostat, 2012). This increased diversity within national 
borders presents a set of challenges and opportunities for the nation-state. For example, some of 
the negative discussion around-increased diversity and immigration, both in public discourse and 
in the media, raise societal anxiety over the relationship between immigration, particularly of 
Bauman’s “vagabonds,” and the social and economic problems perceived to be associated with 
global processes (Appadurai, 2006). Europe is an illustrative example. As internal boundaries 
have become more porous, public debate and anxieties about external borders and immigration 
increase. Beck and Grande (2007) discussed this paradox, in which they argued that the more 
notions around a “traditional” or “insider’s” Europe are dismantled, “the more persistently they 
are invoked and strengthened in relation to Europe’s others” (p. 182). 

Related to issues around immigration and changing demographics, most societies have con- 
tinued to experience deepened economic and social inequalities (Sen, 2001), in spite of global 
mandates focused on poverty eradication over the past several decades (World Bank, 2011). And 
although efforts since the 1980s have been moderately successful, within-country inequalities 
have increased in several large countries, including the United States, China, and India (Dollar, 
2005). Globally, the picture of poverty is staggering: Slightly less than half of the world is living on 
less than $2.50 a day, with at least 80% living on less than $10 a day (World Bank, 2008). Poverty 
is a significant problem in both wealthy and poor countries alike. For example, the United States, 
one of the wealthiest countries in the world, has one of the highest rates of poverty among Organ- 
isation for Economic Co-operation and Development (OECD) countries (OECD, 2012). Further, 
according to the National Center for Homeless Education (2012), there were more than 1 million 
homeless children in the U.S. school system in 2011 posing unique challenges for education. 


Social Cohesion 


Although our previous discussion above suggests a world that is considered both more global and 
increasingly mobile, itis still, in the words of Appadurai (2006), a world dominated by “vertebrate 
structures,” the nation-state among the most pervasive of these structures. The confusion of living 
in a globally mobile world that remains closely tied to the nation-state brings a range of issues for 
those nation-states. For example, how does the nation-state encourage participation of its citizens 
in the global economy and foster good stewardship of a global environment while nurturing loyalty 
to the nation among its population? Add massive global flows of people to this equation and the 
nation-state must work harder than ever to build cohesion among its increasingly diversifying 
and in many cases, persistently unequal citizenry. 

Helpful for framing the nation-states’ need to build social cohesion and legitimacy of its 
functions, we rely on Claus Offe’s (1984) theory of the state, in which he argued that there 
are two primary and mutually reinforcing concerns of state function: accumulation (the state’s 
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management of its economic infrastructure) and legitimation (the state’s management of its 
citizens by gaining and maintaining citizen loyalty, which in turn helps to avoid economic crisis). 
From this perspective, there exists a social contract! between individuals and the state, in which 
state functionality requires buy-in from individual citizens. Education systems play an important 
role in both the state functions of accumulation and legitimation, providing what Green (1997) 
described as “disciplined workers” and “loyal recruits” (p. 134). Here we are most interested in 
legitimation processes, particularly the role of education systems in building cohesive national 
societies and helping to build state legitimacy (Green, 1997; Keating, Ortloff, & Philippou, 2009, 
p. 146). In this way and against the dynamics of globalization related to increased immigration 
and deepened inequalities, countries and international policy organizations alike (e.g. OECD, 
EU, World Bank) have focused on social cohesion as a way of fostering belonging while reducing 
marginalization and fragmentation. 

Despite its frequent use in the policy literature at national and international levels, the concept 
of social cohesion is often only loosely defined. One reason for the lack of a precise definition 
is that the purposes of social cohesion are often dependent on the goals of the organization. 
For example, the World Bank (n.d.) posited that social cohesion is “fundamental for societies 
to progress toward development goals.” To the World Bank (2009), social cohesion involves 
“strengthening communities” by getting citizens to work together to “resolve differences in a 
civil, non-confrontational way.” The World Bank sees social cohesion as a way of lowering 
violence in countries where violence impedes development and focuses on improving social 
cohesion by enhancing people’s attachment to communities. As another example, the EU has 
emphasized social cohesion as a central goal, although it has no exact or even agreed upon 
definition, leaving the concept clouded with ambiguity (Hulse & Stone, 2007). Social cohesion, 
nonetheless, is well integrated in the European social model. It is explicitly mentioned in all 
three pillars of the EU’s Lisbon Strategy—social, economic, and environmental—which aimed 
to make Europe “the most competitive and dynamic knowledge-based economy in 2010 with 
more and better jobs and greater social cohesion” (Commission of the European Communities, 
2000). It is also well integrated into the Europe 2020 Education and Training objectives, which 
emphasize the role that education plays in building a more competitive and socially cohesive 
Europe (Commission of the European Communities, 2010). For both the World Bank and the 
EU, social cohesion is often mentioned as a broad rationale for the pursuit of their respective 
institutional goals, yet often a concrete definition is not provided. 

Following Chan, To, and Chan (2006) and Koonce (2011), we posit that more narrow definitions 
of social cohesion than those used by large organizations are useful in utilizing social cohesion 
for analyses of societies. In the academic literature, one of the more clearly stated definitions of 
social cohesion is by Green, Janmaat, and Han (2009) who characterize social cohesion as “the 
property by which whole societies, and the individuals within them, are bound together through 
the action of specific attitudes, behaviors, rules and institutions, which rely on consensus rather 
than pure coercion” (p. 19). In this definition, which is typical of the literature on social cohesion, 
the term refers to national cohesion, where societal unity is defined at the level of the nation-state 
(Koonce, 2011). Furthermore, as a process, rather than an end goal or condition, social cohesion 


'As discussed in Heyneman (2002-03), a social contract extends beyond a legal contract. It includes a “willingness 
to pay taxes and fulfil other public obligations; it may include the willingness to participate in public affairs, maintain 
cleanliness of one’s property, act responsibly, or be a good citizen” (p. 75). 


GLOBAL MOBILITY AND RISING INEQUALITY 127 


requires “a sense of commitment, and a desire or capacity to live together in harmony” (Jenson, 
1998, p. 1). 

In determining characteristics of a socially cohesive society, scholars have pointed to different 
factors, including a measure of trust among citizens of one another and of society at large, as well 
as an agreement that citizens will cooperate and participate in society, its civic institutions, and 
the existing social norms (Chan et al., 2006; Han, Janmatt, Hoskins, & Green, 2012; Heyneman, 
2005; Hulse & Stone, 2007; Koonce, 2011). One could reasonably argue then, that socially 
cohesive citizens will participate in civic activities (i.e. voting) and generally feel a certain degree 
of trust in their resident country, in turn contributing to state legitimacy (Offe, 1984). 

Beauvais and Jenson (2002) identified several interactive elements that are associated with a 
highly socially cohesive society, including common values and a civic culture, social order and 
social control, social solidarity and reductions in wealth disparities, social networks and social 
capital, and territorial belonging and identity. These interactive elements were built on previ- 
ous work by Jenson (1998) that sought to associate social cohesion with the following spectra: 
affiliation/isolation, insertion/exclusion, participation/passivity, acceptance/rejection, and legit- 
imacy/illegitimacy. Further, studies of social cohesion often explore attitudes and perceptions 
of individuals and groups, such as one’s immigrant background (Spoonley, Peace, Butcher, & 
O’Neill, 2005). For example, Han et al. (2012) studied how the perceptions and experiences of 
inequality by citizens impact factors pertinent to social cohesion, including civic participation, 
tolerance, and trust. Moreover, a number of studies have shown various factors that promote a 
socially cohesive society: involvement/participation within the community, inclusion or influence 
over social decisions, recognition of an individual’s or group’s dignity and legitimacy (Jenson, 
1998; Spoonley et al., 2005). 

In addition to these factors, Jenson and Saint-Martin (2003) discussed how investments in 
poverty reduction and increased resources for children are key elements in the promotion of a 
socially cohesive society. A small but growing body of literature has linked SES with social 
cohesion (Becares, Stafford, Laurence, & Nazaroo, 2011; Han et al., 2012; Kawachi, Kennedy, 
Lochner, & Prothrow-Stith, 1997; Ratcliffe & Newman, 2011). Studies have shown that greater 
income equality is linked with higher levels of social cohesion (Ratcliffe & Newman, 2011; 
Wilkinson & Pickett, 2010). Specifically, in their study, Wilkinson and Pickett’s study (2010), 
they found a strong relationship between trust, often considered a characteristic of social cohesion, 
and income inequality. As one example, the countries where high percentages of citizens agreed 
with the statement that “most people can be trusted” were more income equal countries (Wilkinson 
& Pickett, 2010). Likewise, Green et al. (2006) linked educational equality with greater social 
cohesion. 

Despite the growing body of academic and policy literature on social cohesion, the lack 
of clarity in narrowly defining the specific factors thought to contribute to social cohesion is 
most likely a result of a number of problematic terms such as what it means to be a “demo- 
cratic” “citizen” in a “global” “modern” “society.” Nonetheless, it seems to be a central goal 
of most to create a society where the well-being of all members is pursued without marginaliz- 
ing any members. Although defining the factors of a socially cohesive society may be difficult 
(Koonce, 2011), we contend that there is a great deal to learn by exploring some of the fac- 
tors that are believed to indicate a more socially cohesive society. Specifically, and guided by 
the aforementioned literature, we contend that (a) some useful proxies of social cohesion in- 
clude trust in civic institutions, attitude toward resident country, and value of civic behavior 
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and (b) these indicators are important factors to most educational systems worldwide. To that 
end, of the resources that nation-states can and do provide to children, universal education is 
certainly one of the most prominent. Heyneman (2010) argued that education can contribute to 
social cohesion in four ways: (a) by teaching students the rules that govern interpersonal and 
political action, (b) by providing students with an experience consistent with local citizenship 
principles, (c) by providing equal opportunities to all students, and (d) by incorporating the inter- 
ests of diverse communities while providing students with a common understanding of citizen- 
ship (http://education.stateuniversity.com/pages/2428/Social-Cohesion-Education.html). Social 
cohesion appears to have increased in importance in education partly because of the instability 
and fragmenting nature of globalization trends related to increased cross-border migration as 
well as rising social and economic inequalities. Yet, despite the influx of immigrant students into 
many education systems worldwide, there remains a lack of knowledge about the experiences 
and attitudes of immigrant students compared to their native-born peers (Suarez-Orozco, 2001; 
Sudrez-Orozco et al. 2010). There is also a need to consider the relationship between immigration 
and SES, to better understand the ways in which education systems can respond to immigrant 
populations. Therefore, we contend that there is much to be gained from exploring the relationship 
between immigration, inequality, and the factors that are thought to indicate a socially cohesive 
society (trust in civic institutions, attitude toward country, and value of civic behavior). 

Against this background, we explore the international relationship between immigrant status 
of students and indicators of self-reported social cohesion, and whether that relationship is 
associated with the student’s SES. Our main research questions can be stated as follows: Is there 
a relationship between immigrant status and social cohesion of eight-grade students around the 
world? And if so, does that relationship depend on SES? 


METHODS 
Data 


The International Civic and Citizenship Study (ICCS) 2009 is a survey that examined the ways 
in which educational systems prepare their young people to undertake their roles as citizens 
(Brese, Jung, Mirazchiyski, Schulz, & Zuehlke, 2011). As a main emphasis, ICCS measured the 
conceptual knowledge of civic and citizenship education of more than 140,000 eighth graders 
in 38 educational systems around the world. In addition to achievement measures, ICCS was 
also designed to collect data pertaining to student dispositions and attitudes toward civic and 
citizenship education (Schulz, Fraillon, Ainley, Losito, & Kerr, 2008). The resultant database 
provides extensive, internationally comparable information on civic and citizenship learning 
outcomes. The target population included eighth-grade students who were approximately 14 years 
of age. See the appendix for a list of participating countries, sample sizes, and descriptive statistics. 


Measures 


Based on our research question and in line with our theoretical framework surrounding issues of 
social cohesion, we chose a number of measures from the student questionnaire. We discuss each 
of these in turn. 


GLOBAL MOBILITY AND RISING INEQUALITY 129 


Student Measures 


As a measure of immigrant status, we classified students into three possible categories: native, 
child of an immigrant, or first-generation immigrant. Native status means that both the student 
and his or her parents were born inside the country of the test. Students who were born in the 
country of the test and had at least one parent born outside the country of the test were considered 
children of immigrants. Students who, along with both parents, were born outside the country 
of the test were considered first-generation immigrants. The remaining variables used in the 
current analysis are derived scales, the psychometric properties of which were examined by the 
ICCS project analysts, who made the variables available as part of the international database. We 
provide a brief description of each scale; however, full details for each of these scales, including 
item parameters and model fit can be found in the ICCS Technical Report (Schulz, Ainley, & 
Fraillon, 2011). 

To examine the effect of student-level socioeconomic differences, we included an index 
of socio-economic background (NISB). The NISB scale is a weighted composite of highest 
occupational status of parents, highest educational level of parents, and the number of books 
in the home. This scale has a mean of zero and a standard deviation of 1, with an estimated 
country-level reliability of .50 to .70 as measured by Cronbach’s alpha (Schulz et al., 2011). 
Higher values on the NISB scale correspond to higher SES. Each of our outcome variables are 
weighted composites and include the degree to which students have trust in civic institutions 
(INTRUST), their attitude toward their country of residence (ATTCNT), and their value of civic 
behaviors (C/ITCON). All of the outcome variables are scaled within each country to a mean of 
50 and a standard deviation of 10. The INTRUST scale comprises six items that ask students 
how much they trust the national government, the local government, the courts, police, political 
parties and the national parliament or congress. The ATTCNT scale comprises seven items 
that ask the degree to which the student agrees with statements about their resident country’s 
political system, the flag, and their respect and admiration for the country they live in. Finally, the 
CITCON scale comprises six items that ask students how important a number of behaviors are 
for being a good adult citizen including voting, joining a political party, learning the country’s 
history, following political issues, showing respect for government representatives, and engaging 
in political discussion. The pooled international scale reliabilities for each outcome variable are 
.84, .82, and .71, respectively. 


Analytic Approach 


Multilevel models (e.g., Raudenbush & Bryk, 2001) were fit to the ICCS 2009 data set to answer 
the research questions regarding the relationship between immigrant status, SES, and social 
cohesion. In particular, a model that treated students as nested within schools and schools as 
nested within countries was fit to each of the three outcome variables. Sampling weights were 
applied to all analyses to account for the fact that samples were not drawn at random from the 
population (Rutkowski, Gonzalez, Joncas, & Von Davier, 2010; Schulz et al., 2011) and weights 
were normalized so that the sum of the weights was equal to the sample size. 

As an initial step to provide justification for a multilevel modeling approach, a null or empty 
model was fit to each of the outcome variables. This model contains no predictor variables 
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and serves to decompose the total variance in the outcomes into a within-school part (a7), a 
between-school, within-country part (1) and a between-country part (g6). The proportion of 
variance attributable to each level is just the ratio of that variance parameter over the total 
variance, where the total variance in the outcome is given by o7 + ¥% “ Qo: A general rule 
of thumb indicates that if 5% or more of the variance in the outcome is attributable to the 
between parts of the model (school and country), a multilevel modeling approach is prudent 
(Hox, 1998). 

To answer the research question, the following model was fit to the data using the SAS MIXED 
procedure (SAS, 2003) for each of the outcome variables. 

Level 1% 


OUTCOME: jx = Bojx + Bi jx( FIRST, jx) + Bo jx(CHIMM jx) + B3jx(SE Sijx) 
+ Bq jx (FIRST; jx * SES; jx) + Bs jx (CHIMM; j, * SE Sijx) + Rije 


Level 2: 
Boj = Sook + Soi (MFIRST x) + 802% (MCHIMM j,) + 503« (MNISB jx) + Uo je 
Bi jk = 510% 
Bojk = 520% 
B3jk = 630% 
Ba jk = 540K 
Bs jx = S50 
Level 3: 


Sook = Yo00 + Voor 
doik = Yoro + Voir 
do2k = Yo20 + Vorx 
do3k = Yo30 + Vorx 
310k = Yi00 + Viox 
620k = Y200 + Vr0x 
830k = ¥300 + V30x 
640k = Ya00 + Vaox 
d50x = 500 + Vsox 


According to this model specification, there are level-one effects for first-generation immigrant 
and child of an immigrant status (7199 and y299, respectively), SES (7300), and interactions between 
immigrant status and SES (y4o9 and yso9). Each of these level-one effects varies randomly across 
countries, with country-level error terms denoted as Vpox (h = 1,..., 5). The school-level random 
intercept (Bo;,) is predicted by the overall intercept for country k (do0x.), School-level effects for 
the proportion of first-generation immigrants (69),), the proportion of students who are children 
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TABLE 1 
Variance Components From Null Model 

ee Trust in Civic Institutions Attitude Toward Country Value of Civic Behavior 

ode Sass So EE - a _—_. 
Variance Component Estimate* Proportion? Estimate* Proportion? Estimate* Proportion 
Country (9) 10.28 0.10 13.46 0.14 10.59 0.11 
School (3) 5.14 0.05 6.69 0.07 Bree 0.03 
Student (c*) 83.76 0.84 79.58 0.80 86.46 0.86 
Total 99.18 1.00 wie 1.00 100.45 1.00 


“Because the outcome variables are standardized to a mean of SO and a variance of 100, the variance component 
estimate will almost identically match the associated proportion of variance attributable to a particular level. The 
individual proportions do not sum to zero due to rounding error. 


of immigrants (do2%), the school-level average of SES (d93,), and a random effect, Uoj,, that 
allows the intercept to vary randomly between schools within a country. The country intercept 
(Soox) is predicted by an overall international intercept (7oo0) and a random country effect (Voox). 
Similarly, all of the level-two effects are a function of an overall coefficient (yoro, 7 = 1, ..., 3) 
and a random term (Vo,-,, 7 = 1, ..., 3) that allows for random differences between countries 
for each of these effects. The random components have associated variance parameters of 07, 
7 at the student and school level, respectively. Finally, each country-level random error has 
associated variance components denoted as Lio or Loy with h and r as previously defined, plus a 
country-level intercept variance component denoted as oe The school- and country-level variance 
components express the degree to which an effect varies across schools (in the case of level-one 
intercepts) or countries (in the case of level-two intercepts and all level-one and -two fixed 
effects). 

Based on the model specification just described, parameters for the effect of immigrant status 
estimate the difference in average achievement for first-generation immigrants compared to 
native-born students (71099) and for children of immigrants compared to native-born students 
(200). The model does not explicitly test for differences between first-generation immigrant 
students and students who are children off immigrants. To examine whether there are differences 
in this regard, we add a linear contrast of means to our model that tests whether yj090 = 7200- 
Equivalently, then, the null hypothesis associated with this test is 


Ho : Yi00 — ¥200 = 9 


RESULTS 


Before proceeding to the main model results, we first briefly discuss the results of fitting null mod- 
els to each of the outcome variables. The estimated variance components for each outcome variable 
can be found in Table 1. The proportions of variance in trust in civic institutions attributable to the 
country, school, and individual-level were .10, .05, and .85, respectively. The results for the atti- 
tude toward country scale indicated that variance in the outcome was distributed in the following 
proportions: .14 between countries, .06 between schools within countries, and the remaining .80 
was attributable to between-student differences. Finally, the proportion of variance in value of 
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TABLE 2 


Model Results 
Pei SS efit ce SN a oe 2 ee ee ee eee 


Coefficient (SE) 








Level 1 (Student) 


Trust in Civic Attitude Toward Value of Civic 
Fixed Effect Institutions Country Behavior 
ets ay gt RW es ie 8 ee Se a ne Oe a) ee ee ee re eee 
Intercept 50.30* (0.54) 50.85* (0.61) 49.81* (0.57) 
Student Level 
Ist Gen Immigrant —0.41* (0.33) —2.99* (0.36) 0.26 (0.42) 
Child of Immigrant —0.66 (0.15) —2.10* (0.18) —0.11 (0.16) 
SES 0.10* (0.08) —0.14* (0.06) 0.53* (0.06) 
Ist Gen x SES —0.44* (0.20) —0.29 (0.23) —0.77* (0.22) 
Child of Immigrant x SES —0.31* (0.09) —0.12 (0.11) —0.58* (0.11) 
School Level : 
Proportion Ist Gen Immigrant —0.55* (1.08) —1.65* (0.70) 1.56 (0.83) 
Proportion Children of Immigrants —0.33 (0.47) —2.87* (0.99) 0.31 (0.67) 
M SES —0.56* (0.25) —1.49* (0.17) —0.52* (0.17) 
Random Effect / Variance Component 
Level 3 (Country) 
Intercept 10.94* (2.59) 13.54* (3.22) 11.91* (2.79) 
Ist Gen Immigrant 2.16* (0.94) 2.90* (1.07) 4.31* (2.18) 
Child of Immigrant 0.43* (0.17) 0.81* (0.26) 0.56* (0.22) 
SES 0.17* (0.05) 0.10* (0.03) 0.11* (0.03) 
Ist Gen x SES 0.45 (0.32) 0.80 (0.43) 0.57 (0.41) 
Child of Immigrant x SES 0.05 (0.06) 0.20 (0.11) 0.13 (0.10) 
Proportion Ist Gen Immigrant 13.91 (8.49) 0.76 (2.54) 5.21 (4.32) 
Proportion Children of Immigrants 22) 24.80* (7.38) 8.56* (3.59) 
M SES 2.03* (0.54) 0.82* (0.26) 0.88* (0.26) 
Level 2 (School) 
Intercept 4.01* (0.15) 4.20* (0.15) 2.78* (0.13) 


Residual 82.34* (0.33) 77.66* (0.31) 84.65* (0.34) 
Linear Contrast of Means / Testing yj99 = 7200 
F(1, 37) 0.50 503" 0.70 
Fit Statistics 
—2 Log likelihood 960015.8 951434.3 973139.1 





Note. 1st Gen = first generation; SES = socioeconomic status. 
4p =3205: 


civic behavior at each of the three levels was .11, .03, and .86 at the country, school, and student- 
levels, respectively. In each case, the variation attributable to between-country and between-school 
differences was sufficient to justify a multilevel modeling perspective (Hox, 1998). 

We turn now to the results for fitting the full, three-level models to each of the outcome 
variables. The parameter estimates for all three models can be found in Table 2. 
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Trust in Civic Institutions 


Based on the model results, first-generation immigrants have trust in civic institutions that is 
similar to their native-born peers Pio) = —.41, t (37) = —1.25, p = .22. The same is not true of 
students who are children of immigrants. Instead, these students tend to score lower on the trust 
scale than native-born students 299 = —0.66, 95% confidence interval (CI) = (—0.95, —.37). 
And based on the linear contrast of means, first generation immigrant students and children 
of immigrant students are no different in their trust of civic institutions F (1,37) = .50, p = 
.48. Further, the effect of SES is not statistically significant ~399 =0.10, t (37) = 1.24, p = .22; 
however, when the main effect for SES is considered in conjunction with the interaction term, we 
find evidence that, for higher levels of SES, students with an immigration background score lower 
on trust scale Pago = —0.44, 95% CI = (—0.84, —.04); Pso0 = —0.33, 95% CI = (—0.50, —.13). In 
other words, immigrant students from higher socioeconomic backgrounds tend to have less trust in 
civic institutions than native-born students of similar SES, and also less trust than students with an 
immigrant background from lower SES. At the school level, there was no association between the 
proportion of students from an immigrant background in the school and trust in civic institutions 
Poe 055. 067) 50p pH 62; whore = 20:3387'0 (37). S69) p) = SOnInecontrasisithe 
higher the average SES in the school, the lower the average trust in civic institutions Yo39 = —0.56, 
95% CI = (—1.05, —.07). And according to the results for the variance components, the following 
parameters vary across countries: the average value of trust in civic institutions, the effect of being 
a first-generation immigrant or the child of an immigrant, and the effect of SES at both the student 
and school level. Further, the average value of trust in civic institutions varied across schools 
within a country. 


Attitude Toward Country 


According to the results for the outcome variable attitude toward the student's resident country, 
. several findings emerged. In particular, both first generation immigrant students and students who 
are children of immigrants have poorer attitudes toward their resident country than native-born 
students 299 = —2.99, 95% CI = (—3.69, —2.28); ~ro0 = —2.10, 95% CI = (—2.45, —1.74). 
Further, the size of these differences is notable: immigrant students tend to have scores on this 
scale that are 30% of a standard deviation” lower than native-born students whereas children of 
immigrants score 20% of a standard deviation lower than native-born peers. The linear contrast 
of means also suggested that first-generation immigrant students and children of immigrants 
differed in terms of their attitudes F (1, 37) = 5.03, p = .03, with first-generation students scor- 
ing lower on the attitude scale. In addition, there was a small negative effect for SES 90 = 
—.14, 95% CI = (—0.26, —0.18); however, the interaction effects for immigrant background 
and SES were not statistically significant Y4o0 = —.29, t (37) = —1.27, p = .21; Yso0 = —.12, 
t (37) = —1.09, p = .28. All school-level effects were statistically significant Poo = —1.65, 
95% CI = (—3.02, —0.28); foo = —2.87, 95% CI = (—4.80, —.94); Po3o = —1.83, 95% 
CI = (—1.83, —1.16). Taken together, we can interpret these school-level findings such that 


2Recall that the outcome variables are scaled to a mean of 50 and a standard deviation of 10. 
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higher average SES and higher proportions of students from immigrant backgrounds are predic- 
tive of poorer average attitudes toward the resident country. The estimated variance components 
suggest that average attitudes, the student-level effect of being from an immigration background, 
the effect of SES at both the student- and school-level, and the effect of the proportion of children 
of immigrants all vary across countries. Finally, average attitudes vary across schools within a 
country. 


Value of Civic Behavior 


The last model fit to the ICCS data examined the outcome variable value of civic behavior, the 
findings of which we discuss presently. According to the results, we found that both first generation 
immigrant students and children of immigrants tend not to differ in terms of their value of civic 
behaviors such as voting and belonging to a political party Yjo9 = .26, t (37) = 0.63, p = .54; 
P20 = —0.11, t (37) = —0.67, p = .50 when compared to their native-born peers. Further, there 
are no differences between first generation students and children of immigrants on valuing civic 
behavior (F (1, 37) = 0.70, p = .41). We found evidence of a positive effect for SES ~3o9 = .53, 
95% CI = (0.40, 0.65); however, being from an immigrant background moderates that effect 
Yao0 = —.77, 95% CI = (— 1.19, —0.34); Yso9 = —.58, 95% CI = (—0.79, —0.37). Taken together, 
these findings suggest that the relationship between immigrant status and value of civic behavior 
depends on SES. Although there are no differences between native and immigrant students from 
average SES backgrounds, differences emerge at both the high and low ends of the SES scale. 
Specifically, native students tend to have a greater value of civic behavior at higher ends of 
the SES spectrum. In contrast, poorer students from an immigration background tend to value . 
civic behavior more than similarly poor native students. At the school-level, the proportion of 
children from an immigrant background was not associated with average scores on the value 
scale Poio = 1.56, t (37) = 1.88, p = .07; Yor) = 1.56, t (37) = 0.46, p = .64; and average SES 
was negatively associated with valuing civic behavior yo39 = —.52, 95% CI = (—.86, —.18). The 
estimated variance components provide evidence that average scores on this scale vary across 
country, as do the effects of being a child of an immigrant, SES, and the proportion of students 
from immigrant backgrounds. At the school level, average scores on this scale vary across schools 
within a country. 


DISCUSSION AND CONCLUSION 


As societies become more diverse and people increasingly traverse national boundaries, countries 
consider it important to foster environments where residents are bound together through specific 
attitudes, behaviors, and rules rather than forcing notions of citizenship through pure coercion. 
In other words, nations depend on establishing and maintaining some sort of national identity to 
hold an increasingly diverse citizenry together. This effort is akin to the legitimizing process that 
Offe (1984) described as essential to the nation-state’s well-being. In this article we have argued 
that the introduction of immigrant populations, partly accelerated by global forces, has made 
the nation-state’s goal of integrating populations increasingly difficult and that many nations and 
international organizations have turned to the concept of social cohesion as a way of mediating 
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these processes (Hulse & Stone, 2007; OECD, 2011). Particularly in a context of growing global 
poverty, “strengthening social cohesion [has become] a critical policy objective” for governments 
around the world (OECD, 2011, p. 19). Although the literature has shown that some systems 
appear to perform better than others at building social cohesion (Heyneman, 2000), precisely how 
a nation-state becomes more socially cohesive is not well understood. Despite the lack of definitive 
factors associated with social cohesion, we have argued that many researchers and policymakers 
see schools as locations to promote a more socially cohesive society, often looking to these social 
institutions as spaces to promote ideas related to social cohesion, including increased trust in 
civic institutions, attitude toward the resident country, and value of civic behavior. 

Our literature review also noted that current research has reported little international evidence 
on how immigrant students view ideas around social cohesion compared to their native-born peers. 
In response to this lack of understanding and consensus, we attempted to examine how students 
with an immigrant background differed in terms of attitudes and behaviors that align with ideas 
of social cohesion in comparison to their native peers. Further, in examining the link between 
immigration status and social cohesion, we explored whether differences in SES moderated the 
relationship. At the international level, our findings suggest that there are, in fact, dissimilarities 
among student groups and that it may be necessary to focus interventions between native-born 
and nonnative-born students. But the story is much more complex than simply looking at where 
a student was born. In fact, the SES of the student also appears to play an important role in how 
a student feels about the country where they are living. This finding is in line with early and 
important ideas of globalization such as those of Bauman (1996), who suggested that immigrant 
populations differ greatly and can be further differentiated by their SES, coining immigrant 
groups either tourists or vagabonds. As discussed earlier, tourists tend to navigate borders with 
ease, often being accepted within communities, whereas vagabonds are seen as a drain on social 
systems even though they may provide necessary labour for their host country. 

This important distinction between high SES and low SES immigrants demonstrates an added 
complexity to the social cohesion conversation and suggests another variable that should be 
attended to in policy discussions. For example, our findings show that the difference between 
immigrant and native students’ trust in civic institutions changes significantly as SES increases. In 
particular, trust in civic institutions diminishes for children with an immigrant background as SES 
increases. Although there is no difference overall between native-born students and their foreign- 
born peers’ value of civic behavior, this relationship is moderated by SES. This finding supports 
Bauman’s distinction and suggests that other factors along with immigrant status should be taken 
into consideration when focusing policies at improving social cohesion and the integration of 
diverse student populations in schools. 

Our findings related to the impact of SES on social cohesion among native and immigrant 
students then point to the importance of clarifying and broadening definitions of social cohesion. 
In some of the World Bank and OECD’s work in the 1990s on social cohesion, poverty reduction 
and SES were central to the discussions taking place on social cohesion (Narayan, 1999; OECD, 
1997, 1998). Further, within recent policy literature on social cohesion, a narrower association 
has been made between social cohesion and racial and ethnic identity, and in some cases (e.g. 
England), religious identity (Cantle, 2005; Commission on Integration and Cohesion, 2006; En- 
gel, Kington & Mleczko, 2013). According to Appadurai (2006), this focus is part of the growing 
societal anxiety around ethnic minority groups associated with some of the by-products of global- 
ization: increased immigration, inequality, and exclusion. In many countries, policymakers have 


\ 


136 L. C. ENGEL, L. RUTKOWSKI, AND D. RUTKOWSKI 


increasingly responded to the heightened societal anxiety on the rise of immigrant populations, 
particularly those considered of “vagabond” status. Using Appadurai’s words, this “fear of small 
numbers,” brings yet another challenge to school systems tasked with managing their increas- 
ingly diversifying populations and using education as a means for greater social cohesion. If one 
of the goals of schooling is to increase socially cohesive attitudes and behaviors, then broader 
notions of social cohesion that also consider SES, rather than emphasizing only race, ethnicity, 
or immigration status alone, become important. 

Recent events such as the Boston Marathon bombing in the United States, where naturalized 
citizens allegedly performed these horrible acts, illustrate the complex relationship between 
immigration, societal anxiety over culturally diverse populations, issues of marginalization, and 
violence. Events such as these tend to underscore debates about social cohesion and the importance 
of education, which is considered among the best ways to build social cohesion. Education builds 
social cohesion through communicating social rules, providing experiences with local citizenship 
activities, ensuring equal opportunities to all students, and balancing the interests of diverse 
communities with acommon understanding of citizenship (Heyneman, 2010). In this context, our 
findings suggest the importance of understanding the impact of SES on the attitudes of foreign and 
native-born youth toward their resident country, the value they have on civic behavior and their 
trust in civic institutions. Overall, social cohesion is a slow process involving changing attitudes, 
building trust, confidence, legitimacy, unity, and mutual respect across students and families from 
different cultural and ethnic backgrounds. Schools can serve as powerful institutions to change 
attitudes, foster links between family, school and community, and cultivate trust and confidence 
in Civic institutions. As shown from this study, as national and school-level policies and thinking 
about social cohesion continues to develop, a focus on SES as a way to better understand the 
experiences and needs of immigrant students may be useful. 

Of course, empirical studies that use extant data necessarily suffer from limitations, which we 
note here. First, given the ambiguity surrounding definitions of social cohesion, our operational- 
ization of this term is one of many possibilities. Another choice of variables could have resulted in 
other findings; however, we observed that, regardless of the indicator chosen to represent notions 
of social cohesion, we consistently found differences between native-born students and students 
with an immigration background. Second, the ICCS data are observational, cross-sectional data, 
which limit causal inferences regarding the nature of the relationships. It is worth noting that in 
spite of these limitations, the ICCS data comprised a large sample of diverse countries with differ- 
ent economic, ethnic, and cultural compositions. Even in the face of such diversity, our findings 
point to an international pattern of difference in social cohesion indicators among student groups 
that depends on both immigrant status and SES. We note in closing that large-scale data sets such 
as ICCS (among others) provide a unique opportunity to examine international education policy 
issues and to promote cross-national dialogue among policymakers in an effort to identify possible 
solutions that can be tailored for the unique national context in which these phenomena occur. 
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APPENDIX 


Table of Participating Countries with Sample Sizes and Means on Outcomes 


Countries 


Austria 

Bulgaria 

Chile 

Chinese Taipei 
Colombia 

Cyprus 

Czech Republic 
Denmark 
Dominican Republic 
Estonia 

Finland 

Greece 

Guatemala 

Hong Kong SAR 
Indonesia 

Ireland 

Italy 

Korea, Republic of 
Latvia 
Liechtenstein 
Lithuania 
Luxembourg 
Malta 

Mexico 
Netherlands 

New Zealand 
Norway 

Paraguay 

Poland 

Russian Federation 
Slovak Republic 
Slovenia 

Spain 

Sweden 
Switzerland 
Thailand 

England 

Belgium (Flemish) 


3368 
3243 
5168 
5151 
6185 
3172 
4622 
4444 
4519 
2728 
3289 
3129 
3982 
2873 
5057 
3266 
3342 
5249 
2754 
355 
3892 
4799 
2112 
6519 
1950 
3932 
2969 
3354 
3238 
4285 
2965 
3062 
3286 
3436 
2911 
5246 
2900 
2943 


Attitudes Towards 
Own Country 


51.81 
48.35 
50.13 
46.63 
jot2 7 
49.01 
44.56 
48.45 
56.68 
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We investigate the nature of protests by students (age 18 and older) in Algeria, Jordan, Morocco, 
and Yemen by using subsamples of students from nationally representative and acclaimed public 
opinion data (the 2006-07 Arab Barometer). We find between 22.1% (Jordan) and 54.7% (Yemen) 
participated in either the signing of petitions, or marching in street protests, or both between the years 
2003-07. To explain student protest participation, we draw from the political economy literature to 
test four grievance-based hypotheses that link protest to student perceptions on the performance of the 
economy, personal family socioeconomic status, political exclusion, and preference for democracy. 
Ordered probit regression analyses indicate that students protest for different reasons in the four 
countries. We find statistical evidence that student protests are associated with grievances about the 
economy (Algeria and Morocco) and lack of democracy (Algeria only). Joint hypothesis tests reveal 
that the four grievances jointly matter in Algeria, Morocco, and Yemen but not Jordan. 


INTRODUCTION 


Postsecondary students have threatened or even toppled regimes around the world (Altbach, 
1989). Prominent examples from the 20th century include student protests in Spain against 
Dictator Franco’s rule (1939-1975), in the United States against the Vietnam War (1960s), in 
China’s Tiananmen Square against the Communist Party’s reform efforts (1989), and in South 
Korea’s march toward democracy (1970-1992). In 2010-2011, protests by students and others 
in several countries in the Middle East and North Africa, collectively referred to as the “Arab 
Spring,” transformed the political, economic, and social landscape on a scale that has not been 
witnessed since the fall of communism (Ottaway & Hamzawy, 2011). 

In this study, we investigate the reasons for protests by secondary and postsecondary students 
(age 18 and older) in four states that experienced the Arab Spring: Algeria, Jordan, Morocco, and 
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School of Education, and Ford Center for Human Security. 
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Yemen. We draw from the recent political economy literature arguing that protest participation 
occurs because of economic and political grievances. Using subsamples of students from nation- 
ally representative public opinion data, we test four grievance-based hypotheses that link student 
protest participation to their perceptions on the performance of the economy, personal family 
economic condition, political exclusion, and preference for democracy. 

Unlike previous research on student protest participation in developing regions, we have access 
to highly regarded data: the 2006-07 Arab Barometer. The American Political Science Associa- 
tion recognized the Arab Barometer team by awarding them the 2010 Lijphart/Przeworski/Verba 
Dataset Award for Best Dataset in the field of Comparative Politics. Because the data collection 
preceded the Arab Spring, we can address the origins of one of the least expected and most 
effective student protest movements in recent history. 

We make several contributions to the student protest literature (for a review, see Knutsen, 
2007). First, our study examines student protests in the 2000s, whereas much of the existing 
research has documented protest during the period from 1960 to 1990. Second, we contribute 
to the scarce empirical literature on student protests. Third, our study is comparative, rather 
than focusing on a single country. Finally, most studies have explored institutional or student 
characteristics that influence participation in a specific campus protest. In contrast, we combine 
the student protest literature with the broader political economy literature to examine the broader 
political and economic determinants of protest behavior.! 


CONTEXTUAL BACKGROUND 
Economic Indicators and Political Regimes 


Several observers have argued that weak economies have been responsible for protests and other 
forms of unrest in Arab states (Council on Foreign Relations, 2011, p. 136). We begin our investi- 
gation, therefore, with an overview of the four countries’ economies and systems of government.” 
Comparing the 1991—2000 and 2001-08 periods, economic growth rates improved for Algeria 
(1.74%), Jordan (5.2-7%), and Morocco (2.6-5.1%) but weakened for Yemen (5.6—3.9%). In 
terms of PPP-adjusted income, Algeria ($6,160) and Jordan ($4,629) are upper middle income, 
whereas Morocco ($3,507) and Yemen ($2,428) are lower middle income. Morocco has the 
lowest poverty rate (9%), followed by Jordan (19%), Algeria (22.6%), and the region’s high- 
est rate in Yemen (41.8%). These figures, however, do not reflect the economic volatility that 
arose partly because of the transition from primarily state-supported economies to market-based 
economies. 


‘In particular, our study contributes to the limited microlevel research on political behavior in developing countries 
(Berman, Callen, Felter, & Shapiro, 2011; Jamal & Tessler, 2008). In a review of the literature, Knutsen (2007) only 
listed studies from India and Latin America. Most of the recent research is found in the sociological and political science 
literature but does not focus specifically on student involvement. 

*Figures are for 2008 unless otherwise noted. PPP Converted GDP Per Capita (Laspeyres), derived from growth 
rates of c, g, i, at 2005 constant prices. Alan Heston, Robert Summers and Bettina Aten, Penn World Table Version 7.0, 
Center for International Comparisons of Production, Income and Prices at the University of Pennsylvania, May 2011. 
Site: data.worldbank.org. 
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Politically, Algeria is a semipresidential republic where Abdelaziz Bouteflika came into 
power during emergency rule in 1999. Jordan is a constitutional monarchy that was previ- 
ously ruled by King Hussein (1952-1999) and is now under the reign of his son, King Abdallah 
(1999-present). Morocco is a constitutional monarchy with bicameralism where King Moham- 
mad VI (1999-present) succeeded to the throne after the death of his father, King Hassan II 
(1961-1999). Yemen is a republic where Ali Abdullah Saleh (the president of North Yemen) 
assumed the presidency of the newly unified North Yemen and South Yemen in 1990. The 2011 
Economist Intelligence Unit’s Democracy Index rankings reveal some differences, with Jordan 
(rank 118) and Morocco (rank 119) considered more democratic than Algeria (130) and Yemen 
(150). ral the Arab Spring, regime change occurred in Yemen (Economist Intelligence Unit, 
2011). 


Higher Education and Protests in Algeria, Jordan, Morocco, and Yemen 


The literature on student protests indicates that factors within an education system matter when 
explaining protest participation (Munson, 2010; Zhao, 2009). The Arab Barometer does not 
include data on characteristics of educational institutions and thus does not allow us to statistically 
test their effect in this context. Nonetheless, the anecdotal evidence that follows is consistent with 
the literature.> 

In general, higher education systems in the Middle East and North Africa face significant chal- 
lenges (Heyneman, 1997; World Bank, 2008). The 2010 regional gross enrollment rate average 
was 24%. Research indicates that there is a “reluctance to change and innovate, poor organiza- 
tional frameworks, traditional management systems, financial dependence on governments, and 
lack of autonomy and academic freedom for universities” (Sawahel, 2012). For a population of 
400 million people in the region, there are just 470 universities and institutions currently serv- 
ing 9 million students, with only one in five enrolled in scientific programs (Sawahel, 2012). 
Furthermore, persisting colonial legacies, particularly in North African countries, are at the root 
of problems with access, language of instruction, curriculum, and academic freedom (Teferra & 
Altbach, 2004). 

Gross enrollment rates (GER) for the four countries indicate varying degrees of access to 
higher education. In Algeria, the GER grew from 14% in the 1990s to 31% in 2010 (UNESCO 
Institute for Statistics [UIS], 201 1a). Education accounts for 20.3% of government expenditures, 
with 22% of that going to higher education, including grants for 85% of Algerian students (UIS, 
2011a; European Commission, 2010a). As enrollment rises, Algeria is focusing on increasing 
public funding with plans to “spend US$1.48 billion on higher education and science over the 
next five years and to double research spending to 1% of gross domestic product” (Racelma, 
2012). Despite the growing education sector, students across the country have protested against 


3Bicameralism is a feature of a mixed government that includes the practice of having two legislative or parliamentary 
chambers compromise bills. 

‘In terms of population size, Jordan has the smallest population (6.1 million) followed by Yemen (22.2 million), 
Morocco (30.9 million), and Algeria (33.8 million). A distinguishing characteristic of the populations is the youth bulge: 
the median age ranges from 17.9 years (Yemen) to 26.5 years (Jordan). 

>Because the Arab Barometer subsample of students includes only those who are of age 18 and older, we focus on 
providing a background of the higher education systems. . 
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changes in higher education policy affecting the student experience and employment prospects. 
In 1998, Algerian student unions at universities were involved in strikes against changes in 
university policy, followed by protests in 2000 in the city of Tizi Ouzou by students contesting 
deteriorated physical conditions on university campuses (Madani, 2000). In 2011, students in 
Algiers protested higher education policy they believed devalued university diplomas (“Algerian 
Students Revolt,” 2011). 

Jordan’s GER increased from 22% in the 1990s to 38% by 2010 data (UIS, 2011b). Education 
accounts for 20.6% of government expenditures, with 23% of that going to higher education (UIS, 
2011b). Higher education in Jordan is fee-based with limited financial support for students (Eu- 
ropean Commission, 2010b). Currently, Jordan is promoting the development of its community 
colleges to meet the needs of its growing youth population and address its shortage of workers 
with midlevel vocational skills. Nevertheless, Jordanian students protested in 2000 against higher 
education policy allowing the government to appoint members of university student councils 
(“Jordan’s Students Protest,’ 2000). 

In Morocco there was very little growth in the GER, from 10% in the 1990s to 13% in 2009 
(UIS, 201 1c). Public higher education in Morocco is paid for by the government, which spends 
25.7% of its budget on education, with 15% of that going to higher education (UIS, 201 1c). 
There is, however, a mismatch between acquired skills and labor market needs, leading to protest 
by jobless graduates. Students have also protested against university administrative decisions, as 
was the case in 1984 when students in northern Morocco participated in violent protests against 
increased school and final examination fees and more stringent admissions standards (Randal, 
1984). 

Data and information on higher education in Yemen is somewhat limited. There was a fourfold 
increase in enrollments in the 1990s; although expansion has continued, accounting for popula- 
tion growth, GER has remained stagnant at 10% as of 2010 (UIS, 201 1d; World Bank, 2011). The 
public system is fully state funded, with education accounting for 16% of government expendi- 
tures; the higher education allocation was not reported (UIS, 201 1d). Financial allocations have 
not been able to keep pace with expansion, which has led to increased student-staff ratios (24:1 
to 41:1) and deterioration of educational facilities and resources (World Bank, 2011). Although 
students in Yemen protested raises in school fees in 1999 (“Students Injured in Yemen Clash,” 
1999), protests in Yemen and Jordan have been less prevalent than in Morocco and Algeria. 

In summary, student protests in the four states have been attributed to factors associated with 
higher education institutions (e.g., fees and quality). There is also evidence that students have 
taken to the streets to express economic and political grievances. In the remainder of this study, we 
focus on conceptual and empirical inquiry into the economic and political grievances of students 
and their resulting participation in protests. 


CONCEPTUAL BACKGROUND AND HYPOTHESES 


To understand the concept of protest, it is useful to distinguish it from other possible political 
outcomes. Broadly, political outcomes can be categorized into three sets: attitudes, expertise, and 
engagement (Emler & Frazer, 1999). Political attitudes include tolerance—intolerance, partisan- 
ship, and attitude stability (regarding political identity). Political expertise consists of relevant 
skills (organization, listening, speaking, writing, and collective decision making), propositional 
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knowledge (such as knowledge of the constitution, policy, ideology, current affairs, and history), 
interest in politics, understanding of power and authority, and sense of political efficacy. The final 
category of political engagement includes participation in activities such as voting, contacting, 
campaigning, and protesting.° 

Given that our data are on individuals and not groups, we develop theories that address why 
individual students participate in protest activities. In this section, we draw from the recent 
political economy literature on grievance-based explanations of protest participation. In addition, 
we provide a link between grievance-based theories and prevailing social movements and social 
reproduction-based theories of student protest participation. 


Economic Grievance 


Research has suggested economic grievance can lead to protest participation through mechanisms 
of frustration with macroeconomic policies and personal economic conditions. Dalton, Van Sickle, 
and Weldon (2010) demonstrated the importance of macrolevel economic influences on protest 
participation among the general public, particularly in developing countries, where “personal 
dissatisfaction may represent severe economic deprivation or the struggles to survive” (p. 57). 
Muller, Dietz, and Finkel (1991) studied protest participation among university students in Peru 
and found that relative deprivation of socioeconomic status was a strong predictor for participation. 
Almeida (2007) used newspaper content analysis to examine student protest participation in Latin 
America as the region experienced the structural adjustment policies of the 1990s. He found the 
highest levels of participation among the working and middle classes and concluded, “At the core 
of this new motivational structure were perceived threats to economic benefits” (p. 124). Anecdotal 
evidence from the four states also suggests that students protest because of macroeconomic issues 
such as austerity measures. For example, protest among unemployed students has been common 
in Morocco, including the 1984 protests and the 2008 student protests that shut down the port at 
Sidi Ifni in response to job shortages for graduating students (“Poor Quality Education Threatens 
Stability,’ 2008). We state the hypothesis that relates grievance about the performance of the 
economy and student protest as follows: 


H1: Students protest against their government when they associate their own economic hardship with 
the government’s macroeconomic policies. 


A competing argument is that macroeconomic policies and the overall condition of the econ- 
omy do not drive people to protest; instead, people protest when they personally are not doing 
well. “The correlation between low per capita incomes and higher propensities for internal war is 
one of the most robust empirical relationships in the literature,” although there is no consensus on 
the mechanisms that connect income and conflict (Blattman & Miguel, 2010, p. 4). We refine the 
finding of a connection between conflict and low per capita income (a country-level variable) by 


Protest participation does not necessarily imply benevolent, selfless, or socially conscious behavior; rather, it may 
be that protest participants are mainly concerned with improvements in their own outcomes. It is not possible to decipher 
from the data or the anecdotal country evidence the true individual motives behind protest participation. In our study, it 
is unclear if respondents are mainly concerned with improving their own labor market and family status, or broader labor 


market and family status. 
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investigating the relationship between individual participation in protest and economic wellbeing 
at the student family level. 


H2: Students protest against their government when they believe their families’ economic situations 
are poor. 


Political Grievance 


Political grievance is another prominent and intuitively appealing explanation for opposition to 
the government. It is the perception that one is excluded from, or has limited access to, political 
power (Lake & Rothchild, 1996; Levy & Thompson, 2010, Lijphart, 1977). A great deal of 
student protest in Algeria since the 1980s has been focused on Algerian government policy on 
the adoption of Berber as an official language and treatment of the northern region of Kabylia 
(Magistad, 1987). Limited ability to influence decisions about the distribution of public goods and 
patronage benefits is thought to incite feelings of grievance (Blattman & Miguel, 2010; Langer, 
2005; Reynal-Querol, 2002; Scacco, 2008). Muller et al. (1991) argued that dissatisfaction with 
the “current provision of public or collective goods” was a strong predictor of protest participation, 
based on West German panel data in the 1980s. Among the purported reasons for the Arab Spring 
was “frustration with closed, corrupt, and unresponsive political systems” (Goldstone, 2011, p. 
8). We expect students who protest to feel they have inadequate access to authority figures or too 
little influence on government policies. These arguments about the effect of political grievance 
in the form of political exclusion can be expressed by the following hypothesis: 


H3: Students protest against their government when they are aggrieved due to political exclusion. 


The popular narrative of the Arab Spring asserts that the people took action because they 
wanted democracy (Gause, 2011). General frustration with political exclusion is not considered 
an adequate explanation for protest participation, according to this view, because people are 
not seeking better access within the prevailing system; they want democracy. Gause (2011) 
argued that for many years Arabs have not passively accepted authoritarian rule and have voted 
enthusiastically when given real electoral choices. If protest participation is linked to a desire for 
democracy, we expect student respondents to speak favorably about democracy as a governing 
process for their country. The resulting hypothesis linking political grievance about the lack of 
democracy and student protest is as follows: 


H4: Students protest against their government when they want democracy in their country. 


DATA AND MEASUREMENT 


The data for Algeria, Jordan, Morocco, and Yemen come from the 2006-07 Arab Barometer 
project, which is a collection of nationally representative public opinion surveys. The surveys 
were collected in the 2006-2008 period. The contents of the Arab Barometer are similar to 
public opinion surveys such as the World Values Surveys, and the regional Asian Barometer, 
Afro-Barometer, and Latinobarémetro. According to its key investigators, the Arab Barometer 
project was motivated by the belief that the political attitudes and values of ordinary men and 
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women is important for scientific inquiry and the dispelling the myths and stereotypes that hinder 
mutual understanding and cooperation (Tessler & Jamal, 2006). The key investigators of the Arab 
Barometer project were based at the University of Michigan and Princeton University and joined 
by country team leaders and steering committee mémbers from universities and research centers 
from each of the participating countries.’ 

The original 2006-07 Arab Barometer contains samples of adults of age 18 and older, We 
restrict our analysis to subsamples of individuals who identified themselves as “student.” Re- 
garding education, the Arab Barometer asked all respondents, “Level of education?” with seven 
options: illiterate, elementary, primary, secondary, college diploma—2 years, BA, MA, or higher. 
However, this question is unclear about a respondent’s current or completed level of education; 
for example, if a student responds “BA,” we do not know if she is currently pursuing a BA or has 
already completed her BA. Given this ambiguity, we do not explore differences in student protest 
participation across educational levels. 

We define protest participation as signing a petition, attending a demonstration or protest 
march, or both at least once during the preceding 3 years. In reporting protest activities for 
the past 3 years, respondents were given options of “once,” “more than once,” and “never.” We 
combined the “once” and “more than once” categories because of the relatively small share of 
responses in the “once” category. Table 1 matches our hypotheses with the specific questions we 
use from the Arab Barometer survey to measure the outcome, explanatory, and control variables. 

The outcome variable PROTEST=0 if the student had not participated in any form of protest 
in the past 3 years. PROTEST=1 if the student had signed a petition only once or more than 
once. PROTEST=2 if the student had attended a demonstration or protest march (the student 
may or may not have also signed a petition) in the past 3 years. A higher value of the PROTEST 
variable reflects the riskiness of the protest activity. In other words, we consider not protesting 
to be the least risky activity, and marching in a demonstration riskier than signing a petition 
because protesters are more vulnerable to attacks from the authoritarian regime’s security forces. 
PROTEST is therefore an ordinal variable such that the data are mutually exclusive (i.e., there is 
Just one outcome value for each student), the order of the rank is meaningful (i.e., a higher score 
reflects the riskiness of protest), and the scale values are not equally distant (i.e., PROTEST=2 
does not imply that marching is twice as risky as signing a petition). Given the ordered, discrete, 
and quantitative nature of PROTEST, we use an ordered probit regression model (Long & Freese, 
2006). 

The explanatory variables correspond to our four hypotheses. To measure perception of 
macroeconomic policies, we use a survey question that asks about the respondents’ expecta- 
tion for the country’s economic condition in 3 to 5 years. Our proxy for the student’s personal 
economic situation is a question that asks the respondent to rate his or her family’s current eco- 
nomic condition. Political exclusion is measured with a question that asks respondents to agree 
or disagree that government officials seriously consider citizen’s opinions. To measure the desire 


7The data are publicly available from the official Arab Barometer website (http://www.arabbarometer.org). The quote 
was also obtained from this page (last accessed on October 15, 2012). Three countries from the 2006-07 Arab Barometer 
are excluded in this study: Lebanon, Kuwait, and the Occupied Palestinian Territories (OPT). Lebanon is excluded because 
it is already a democracy. Kuwait is unusually wealthy and has not taken efforts to remove its rulers. OPT was not trying 
to bring down an authority. International news organizations such as the BBC have also not included Lebanon, Kuwait, 
and OPT as countries experiencing protests (http://www.bbc.co.uk/news/world- 12482293) 
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TABLE 1 
Arab Barometer Questions Used to Measure Outcome, Explanatory, and Other Control Variables 


SLE EEE EEE 


Outcome variable: 
jrain es Bie weed io oie eee ee ee Se SSS ee 

Protest participation “Here is a list of actions that people sometimes take as citizens. For each 
of these please tell me whether you, personally, have ever done each of 
these things in the past three years.” Among the statements that were 
read was: “Attend a demonstration or protest march.” (0 = never, | = 
once, 2 = more than once) 

Explanatory variables: 

Economic grievance H1: Students protest against their government when they associate their 
own economic hardship with the government’s macro-economic 
policies. 

Arab Barometer question: “What do you think will be the state of 
[country name]’s economic:condition a few years (3—5 years) from 
now?” (1 = much better, 2 = a little better, 3 = about the same, 4 =a 
little worse, 5 = much worse) 


H2: Students protest against their government when they believe their 
family’s economic situation is poor. 

Arab Barometer question: How would you rate the economic condition of 
your family today? (1 = very good, 2 = good, 3 = bad, 4 = very bad) 


Political grievance H3: Students protest against their government when they are aggrieved 
due to political exclusion. 

Arab Barometer question: Do you agree/disagree with the following 
statements: 3. Government officials seriously consider citizens’ 
opinions? (1 = strongly agree, 2 = agree, 3 = disagree, 4 = strongly 
disagree) 


H4: Students protest against their government when they want democracy 
in their country. 

Arab Barometer question: Tell me what you think about a democratic 
political system that includes public freedom, equal political and civil 
rights, balance of power, accountability and transparency (1 = very 
bad, 2 = bad, 3 = good, 4 = very good) 

Control variables: 


Female 0 = male, | = female 
Age Exact age 
Higher education 0 = secondary education or below, | = higher education 





for democracy, we use a question from the Arab Barometer that asks the student’s opinion about 
a democratic political system. 

We include a number of control variables that are typically included in empirical studies 
of protest participation. A gender dummy variable is included because of the large body of 
comparative literature that indicates that men and women participate politically in different ways 
(Inglehart & Norris, 2003). The age variable permits us to explore whether protest participation is 
related to age. We further include adummy variable to distinguish higher education students versus 
secondary education students. Ideally, we would have liked to also control for characteristics of 
the educational institutions being attended, such as extent of support provided by the state, 
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FIGURE 1 Student protest activity by country, 2004-2007. Source: Authors’ computations based on the Arab Barom- 
eter 2006-07 subsample of secondary and postsecondary students (age 18 and older). The sub-samples sizes (n) for each 
country are as follows: Algeria (n = 191), Jordan (n = 100), Morocco (n = 72), and Yemen (n = 117). 


reputation, number of political organizations, and so on. Because the Arab Barometer does not 
contain such information, there is the possibility that our regression coefficients will suffer from 
omitted variable bias. 


RESULTS 
Descriptive Statistics 


Figure | presents the protest activities of students (age 18 and older) in the four Arab states 
during the 2004—07 period. There are three key findings from Figure 1. First, protest participation 
was common during the years preceding the Arab Spring of 2010-11 in Algeria (44.0%) and 
Yemen (54.7%); in contrast, protest participation was relatively unusual, although not rare, in 
Jordan (22.1%) and Morocco (24.4%). Second, student protest participation rates correspond to 
the extent of historic political unrest and recent leadership described in earlier sections. Third, the 
riskier form of protesting (i.e., marching in a demonstration) was approximately 1.5 to 2 times 
as common as the less risky form of protest (i.e., signing a petition). To put these figures in 
perspective, the protest participation rates for the combined samples of students and nonstudents 
are slightly higher (Seybolt & Shafiq, 2012): Algeria (45.0%), Morocco (33.9%), and Yemen 
(58.5%); this implies that students had lower protest participation rates than nonstudents in all 
four Arab states. 

Table 2 presents the summary statistics of the outcome and explanatory variables for the 
student subsamples in each Arab state. The outcome variable means are consistent with Figure 1: 


Protest participation scores (ranging from 0 to 2) are highest in Yemen and considerably lower 
i 
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TABLE 2 
Summary Statistics of Student Characteristics in Arab States, 2004-2007 
i 
Algeria Jordan Morocco Yemen 
M (SD) M (SD) M (SD) M (SD) 
Outcome variable: 
Protest participation 128 460 458 1.017 
(.870) (.784) (.821) (.900) 
Explanatory variables: 
Economic grievance: Economy 2.681 2.250 2.653 2.889 
(.766) (57) (.906) (.908) 
Economic grievance: Family status De Sieh 2.200 2.194 1.940 
(.640) (.791) (.573) (.620) 
Political grievance: Government responsive 2.953 2.570 3.014 2.658 
(1.012) (.844) (.796) (1.027) 
Political grievance: Prefer democracy 3.215 3.410 3.597 3.128 
(.908) (.621) (S97) (.979) 
Control variables: 
Female 445 .400 541 658 
(.498) (.492) (.502) (.476) 
Age 22.6 21.4 13 22.6 
(229) (3.8) (.33) (3.3) 
Higher education .665 170 AIT 419 
(.473) (.377) (.497) (.495) 


N 191 100 72 Laz, 


Note. The outcome variables is PROTEST (=0 have not protested in last 3 years; =1 if signed a petition in last 3 years; 
=2 if marched in a protest in last 3 years). Age is Algeria is reported as a series of intervals, not years. Source: Authors’ 
calculations based on student subsamples of 2006-07 Arab Barometer. 


in Jordan and Morocco. The mean scores for the grievance variables (ranging from | to 4) reveal 
several patterns. In all four states, the preference for democracy is the main form of grievance. 
Political grievance about government responsiveness received the next highest grievance score in 
Algeria, Jordan, and Morocco. In contrast, economic grievance about family status is associated 
with the lowest grievance scores in all four Arab states. The overall results in Table 2 suggest 
that, on average, student grievances in Algeria, Jordan, Morocco, and Yemen have more to do 
with politics than economics. 

The control variable statistics in Table 2 reveal several patterns about the student populations 
in the four Arab states. The female indicator variable shows considerable variation in student 
gender composition across the four states: relatively smaller shares of female than male students 
in Algeria and Jordan, a slightly larger share of female than male students in Morocco, and a far 
larger share of female than male students in Yemen. As mentioned earlier, the Arab Barometer 
asked students about “level of education” without clarification on the completed or current level 
of education. It is therefore unsurprising to see considerable noise in the responses on higher 
education shares—ranging from 66.5% in Algeria to 17.0% in Jordan. Another reason for the 
differences in higher education shares could be because Jordan has younger respondents (who 
have not yet enrolled or completed higher education); this conjecture is supported by the fact that 
the mean for the age variable is lowest for Jordan in Table 2. 
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TABLE 3 
Ordered Probit Regressions Results of Student Protest Participation in Arab States, 2004-2007 
Algeria Jordan Morocco Yemen 
Coefficient Coefficient Coefficient Coefficient 
(SE) (SE) (SE) (SE) 
Se ee 
Economic grievance: Hoes 049 .432** Soe 
Economy 
(.125) (.201) (.226) (.137) 
Economic grievance: —.129 —.027 = ei2ns —.063 
Family status 
(.147) (.186) (.529) (.206) 
Political grievance: —.150* —.031 .479* ae 
Government responsive 
(.087) (.172) (.273) (.117) 
Political grievance: Prefer .175* .249 =.232 —.123 
democracy 
(.098) (.227) (.292) (.128) 
Control variables: 
Female —.339* .064 —.338 —1.498** 
(.183) (.284) (.381) (.282) 
Age .019 —.029 .879* .009 
(.034) (.040) (.526) (.044) 
Higher education =.033 048 250 oT 
(.208) (.407) (.395) (.264) 
/Cut | 1.047 819 79) —2.761 
(.930) (1.160) (1.843) (1.293) 
/Cut 2 1.539 els a 162 —2.136 
(.932) (1.163) (1.812) (1.285) 
Pseudo R? .040 012 214 150 
N 191 100 72 117 
Wald test: Grievance 
coefficients jointly = 0 
x? 10.39 Oy 10.65 8.76 
Pp .034 866 031 .067 





Note. The outcome variables is PROTEST (=0 have not protested in last 3 years; =1 if signed a petition in last 3 years; 
=2 if marched in a protest in last 3 years). Robust standard errors in parentheses. Source: Authors’ calculations based on 
student subsamples of Arab Barometer. 

** Statistically significant at .05 level. *Significant at .10 level. 


Ordered Probit Regression Results 


The coefficients in Table 3 should be interpreted as follows. A positive and statistically significant 
coefficient provides support for a hypothesis. In contrast, a negative and statistically significant 
coefficient indicates that a hypothesis is rejected for the country in question. A statistically 
insignificant coefficient suggests that there is insufficient statistical evidence in support of the 
hypothesis, holding all other characteristics constant. Furthermore, the coefficients reveal student 
propensities to participate in a riskier form of protest (as discussed earlier, not protesting is least 
risky, petition signing is medium risk, and marching in a protest is high risk). 


\ 
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Column | in Table 3 presents the ordered probit regression results for Algeria. There 1s statis- 
tical support for the economic grievance hypotheses. However, the results indicate that economic 
grievance about the family status is unrelated to protest participation, holding other factors con- 
stant. The findings for political grievances are also mixed. Feelings of political exclusion are 
negatively related to protest participation, holding other characteristics constant, we therefore 
reject the hypothesis that political exclusion is associated with student protest. Preference for 
democracy, however, is associated with a higher likelihood of protest participation among stu- 
dents, holding other grievances and factors constant. The overall results for Algerian student 
protestors are consistent with the narrative that protestors in Algeria have been motivated by 
macroeconomic concerns and democratic reform. 

Column 2 in Table 3 shows the ordered probit regression results for Jordan. Curiously, there is 
no statistical evidence in support of any of the grievance hypotheses. In other words, there is no 
evidence that economic grievances (about the economy and family status) or political grievances 
(about exclusion and preference for democracy) are related to student protest participation in 
Jordan. These findings are unexpected because protests in Jordan have traditionally coincided 
with economic downturns and calls for greater democracy. 

Column 3 in Table 3 presents the ordered probit regression results for Morocco. The results 
suggest that macroeconomic grievance is associated with higher student protest participation, 
holding other grievances and characteristics constant. Surprisingly, grievance regarding family 
economic status is negatively associated with student protest participation. Thus, we reject the 
economic grievance hypothesis that students protest against their government when they be- 
lieve their family’s economic situation is poor. This inverse relationship between low family 
economic condition and student protest may be attributable to poor Moroccans feeling de- 
moralized about contributing to political change. Turning to political grievance explanations, 
there is evidence that students participate in protest when they feel politically excluded. Pref- 
erence for democracy in Morocco, however, is not statistically associated with student protest 
participation. 

Last, column 4 in Table 3 presents the ordered probit regression results for Yemen. There is 
no statistical support for the economic grievance hypotheses that students protest against their 
government when they believe their economy or personal economic situation is poor. Contrary to 
the political grievance hypothesis on political exclusion, we find that students who feel politically 
excluded are less likely to participate in protests; we therefore reject the hypothesis on grievance 
of political exclusion and protest participation in Yemen. A likely explanation for this result is 
that politically excluded students feel disenfranchised and helpless in bringing about political 
change. There is also no statistical support for the hypotheses that students in Yemen participate 
in protests against their government because they prefer a democracy. 

So far, our results have indicated that some but not all grievances are statistically associated 
with protest participation. But are all four economic and political grievances jointly associated 
with protest participation? To address this question, we conduct a test of the complex hypotheses 
that involves all the grievance factors set simultaneously equal to zero. According to the Wald test 
results shown at the bottom of Table 3, the hypothesis that the effects of all the grievance variables 
simultaneously equal to zero can be rejected at the .05 level of statistical significance for Algeria 
and Morocco. For Yemen, the joint hypothesis is rejected at the .10 level of significance. However, 
we cannot reject the hypothesis for Jordan, which suggests that broad economic and political 
grievances are not associated with student protests in Jordan. In short, we find that a combination 
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of the four economic and political grievances is associated with student protest participation in 
Algeria, Morocco, and Yemen but not in Jordan. 

Regarding the control variables, there is no evidence of a gender gap in protest participation 
in Algeria, Jordan, and Morocco; in Yemen, however, female students are less likely to protest, 
holding other factors constant. The lack of gender difference is consistent with emerging literature 
on the significant advances in women’s roles in the public sphere (Inglehart, 2003). There is also 
no statistical difference in the protest participation rates of secondary and higher education 
students, holding other characteristics constant. This suggests that secondary students develop 
political views and behaviors prior to enrolling in higher education. As future research, it is worth 
investigating the development of politicization in secondary schools in the four Arab states. 

Although not included in this article, we conducted a series of regressions to assess the 
robustness of our findings. For example, we collapsed students who signed petitions and those 
who marched in one group, such that PROTEST=0 if a student did not protest and PROTEST=1 
if a student either signed a petition, or marched, or both. The findings from the resulting binomial 
regression models are consistent with the ordered probit regression results presented in this article. 


DISCUSSION 


These results suggest the need for additional research. We encourage scholars to assess our 
findings using future rounds of the Arab Barometer. Methodologically, there is broad scope for 
quantitative, qualitative, and case study research that address the effects of both educational 
institutions and broader economic and political grievances on Arab student protest participation. 

We also encourage the adoption of alternative conceptual frameworks for understanding stu- 
dent protest in Arab states. Scholars have often used social movement theoretical frameworks to 
analyze student protest. The bulk of the literature on social movements examines the emergence 
and maintenance of movements from two primary perspectives: resource mobilization and polit- 
ical process. Resource mobilization approaches to social movement analysis focus primarily on 
the role of access to resources in movement emergence. Scholars in this area are interested in 
understanding the ways in which groups utilize available resources—often by leveraging contacts 
with elite groups—and they argue that resources are the key component in movement success and 
maintenance (Jenkins & Parrow, 1977; McCarthy & Zald, 1977, 2002). In contrast, the political 
process model, pioneered by Charles Tilly and Doug McAdam, argues that the emergence and 
maintenance of social movements (such as protests) depends on access to resources and the 
appropriate political and social contexts (McAdam & Tarrow, 2011; Meyer, 2004; Tilly & Wood, 
2009). Accordingly, a social movement conceptual framework would focus on the macrolevel 
processes and organizations involved in student protests rather than on the microlevel individual 
determinants that we investigate in this article.® 


8A number of different approaches to the analysis of social movements have developed out of these two perspectives. 

Some scholars examine the role of culture in social movements (Polletta, 2008; Rochon, 1998; Zhao, 2010), granting 

more agency to cultural contexts in movement emergence and maintenance. Social-psychological perspectives examine 

group behavior, emotional manipulation, and collective identity in movements (Hirsch, 1990; Jasper, 2011; Klandermans, 

2005; Melucci, 1995; Snow & Oliver, 1995). Others study the role of structural determinants in movement mobilization 
\ 
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Educational stratification theory is another alternative conceptual framework for future re- 
search on student protests in Arab states. With roots in the classical Marxist thesis on working-class 
participation in revolutionary social movements, the conflict tradition of educational stratification 
emphasizes an adversarial relationship between classes, where upper classes strive to preserve 
their privilege. Termed “opportunity hoarding” (Wright, 2008), exclusionary practices on the 
part of upper classes manipulate the value of human capital by restricting lower class access to 
education. Wright emphasized the relationship between “the advantages of one group and the 
disadvantages of another,” as opposed to the mobilizing power of human capital at the individual 
level (p. 339). Because university students from richer and financially stable families attend pres- 
tigious educational institutions with higher quality education and better social networks and go 
on to enjoy high status in society (Apple, 1978; Bourdieu, 1973), students from richer families 
do not want to disrupt the status quo by protesting against the regime. In contrast, students from 
poorer families tend to attend lesser quality educational institutions and are relegated to low 
social status after graduation; thus, low-status students are dissatisfied and more likely to protest. 
An inquiry focused on educational stratification conceptual framework would require data on the 
quality and reputation of the educational institutions attended by student respondents to the Arab 
Barometer. 


CONCLUSION 


Using data from the 2006-07 Arab Barometer, we analyzed student protest participation between 
2003-07 in Algeria, Jordan, Morocco, and Yemen. We found protests were common during 
the years preceding the Arab Spring and that the riskier form of protesting (marching in a 
demonstration) was approximately 1.5 to 2 times as common as the less risky form of protest 
(signing a petition), despite the dangers of voicing dissent in authoritarian regimes. 

We proposed grievance explanations of why students protested. In particular, we hypothesized 
that student protest is determined by attitudes on the performance of the economy, personal fam- 
ily economic status, political exclusion, and preference for democracy. Using an ordered probit 
model, we found that student protest participation is statistically associated with grievances about 
the economy (Algeria and Morocco) and lack of democracy (Algeria only). In joint hypothe- 
sis tests, however, we detected support for the four grievance-based explanations for Algeria, 
Morocco, and Yemen but not for Jordan. 

There are two main conclusions to be drawn from our study. First, different motivations 
for student protest are dominant in different countries. Scholars, particularly those who do 
comparative statistical studies, tend to look for a single explanation that holds across multiple 
countries, but our empirical analysis indicates that a more nuanced understanding of student 
political behavior might be appropriate. The second conclusion, based on the lack of statistical 
evidence on individual hypotheses, is that broader economic and political factors may have had 
a limited effect on student protests during the 2003-07 period in Algeria, Morocco, Yemen, 
and especially Jordan. It is possible, however, that student grievances increased during the Arab 


and continuation; for example, organizations (Ganz, 2000; McAdam, 1983), networks (Diani, 2004; Snow, Zurcher, & 
Ekland-Olson, 1980), coalitions (Staggenborg, 1986; von Bulow, 2011), or repression (Alimi, 2009; Almeida, 2003). 
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Spring as the larger population of nonstudents organized mass protests that expressed economic 
and political grievances. 
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